Papers accepted at ACL-IJCNLP 2021 and NAACL-HWT 2021

Congratulations to Dr Frédéric Blain who has had the following papers accepted at upcoming conferences.


Title: Knowledge Distillation for Quality Estimation

Authors: Amit Gajbhiye, Marina Fomicheva, Fernando Alva-Manchego, Frédéric Blain, Abiola Obamuyide, Nikolaos Aletras and Lucia Specia

Abstract: Quality Estimation (QE) is the task of automatically predicting Machine Translation quality in the absence of reference translations, making it applicable in real-time settings, such as translating online social media conversations. Recent success in QE stems from the use of multilingual pre-trained representations,where very large models lead to impressive results. However, the inference time, disk and memory requirements of such models do not allow for wide usage in the real world.

Attempts have been made at making pre-trained representations less resource-hungry by using knowledge distillation, but the resulting models remain prohibitively large for many usage scenarios.Instead of building upon distilled pre-trained representations, we propose to transfer knowledge from a strong QE teacher model to a much smaller model with a different, shallower architecture. In combination with a confidence-based data augmentation approach, we show that it is possible to create light-weight QE models that achieve comparable results to distilled pre-trained representations with 8x fewer parameters.

This paper should appear in the Findings of ACL-IJCNLP 2021 (https://2021.aclweb.org/).


Title: Backtranslation Feedback Improves User Confidence in MT, Not Quality

Authors: Vilém Zouhar, Michal Novák, Matúš Žilinec, Ondřej Bojar, Mateo Obregón, Robin L. Hill, Frédéric Blain, Marina Fomicheva, Lucia Specia, Lisa Yankovskaya

Abstract: Translating text into a language unknown to the text’s author, dubbed outbound translation, is a modern need for which the user experience has significant room for improvement, beyond the basic machine translation facility. We demonstrate this by showing three ways in which user confidence in the outbound translation, as well as its overall final quality, can be affected: backward translation, quality estimation (with alignment) and source paraphrasing. In this paper, we describe an experiment on outbound translation from English to Czech and Estonian. We examine the effects of each proposed feedback module and further focus on how the quality of machine translation systems influence these findings and the user perception of success. We show that backward translation feedback has a mixed effect on the whole process: it increases user confidence in the produced translation, but not the objective quality.

This paper will appear at NAACL-HWT 2021 (https://2021.naacl.org/).

The paper can also be found here: https://arxiv.org/abs/2104.05688

Natural Language Processing

Dr Loïc Barrault, University of Sheffield

17 May 2021

Title: (Simultaneous) Multimodal Machine Translation

Abstract:

Humans perceive the world and interact with it in multimodal ways and language understanding and generation is not an exception. However, current natural language processing methods often solely rely on text to produce their hypotheses. In this talk, I will present recent works aiming to bring visual context to machine translation along with a qualitative assessment of the model capability to leverage this information. We show that while visual context helps under certain conditions, the model tend to be lazy.

Speaker bio[1]:

Loïc Barrault (M) is a Senior Lecturer in the Natural Language Processing group at the University of Sheffield. He obtained his PhD at the University of Avignon in 2008 in the field of automatic speech recognition. Then he did

2 years as researcher and 9 years as Associate Professor at LIUM, Le Mans Université working on statistical and neural machine translation. Loïc Barrault participated in many international projects, namely EuroMatrix+, MateCAT, DARPA BOLT, and national projects, namely ANR Cosmat, “Projet d’Investissement d’Avenir” PACTE and a large industrial project PEA TRAD. He coordinated the EU ChistERA M2CR project and is currently actively involved in the ChistERA ALLIES project. His research work focuses on statistical and neural machine translation, by including linguistics aspects (factored neural machine translation), by considering multiple modalities (multimodal neural machine translation) and by designing lifelong learning methods for MT. He is one of the organisers of the Multimodal Machine Translation shared task at WMT.

[1]: source: https://loicbarrault.github.io/

Paper accepted at ACL-IJCNLP 2021

Congratulations to one of our PhD students Tharindu Ranasinghe – who has had a paper accepted at ACL-IJCNLP 2021.

An Exploratory Analysis of Multilingual Word Level Quality Estimation with Cross-Lingual Transformers. 

Authors: Tharindu Ranasinghe, Constantin Orasan, Ruslan Mitkov

Abstract: Most studies on word level Quality Estimation (QE) of machine translation focus on language-specific models. The obvious disadvantages of these approaches are the need for labelled data for each language pair and the high cost required to maintain several language-specific models. To overcome these problems, we explore different approaches to multilingual word level QE. We show that these QE models perform on-par with the current language-specific models. In the case of zero-shot QE, we show that it is possible to accurately predict word level quality for any given new language pair from models trained on other language pairs. Our findings indicate that the word level QE models based on powerful pre-trained transformers we propose on this paper generalise well across languages, making them more useful in real-world scenarios.

Machine Learning/Deep Learning

Professor Daichi Mochihashi

How LSTM Encodes Syntax: Exploring Context Vectors and Semi-Quantization on Natural Text

11 May 2021

Abstract:

LSTM (Hochreiter 1997) is a well-known recursive neural network that models time series and word sequences, and applied in many fields including natural language processing. For example, Google neural machine translation (Wu+ 2016) is a model that connects 8-level LSTM both for the source language and the target language.

Lately several studies investigate the behavior of neural networks including LSTM; however, they are all limited in their method of regarding neural network as a black-box and only analyze its outputs. On the contrary, in this study we directly investigate the internal vectors of LSTM from a statistical point of view, and visualize how its state vectors distribute.

Specifically, leveraging syntactic tree assigned to Penn Treebank WSJ corpus, we show that the depth of syntactic tree can be highly predictable using a linear regression on some elements of its state vectors. When using only raw words as input, precision becomes lower but the statistical behavior is still the same. We also show that the state vectors associated with polysemous words such as “that” actually distribute differently, which are identified with an ingenious use of PCA.

This is a joint work with Chihiro Shibata (Hosei University) and Kei Uchiumi (Denso IT Laboratory), and appeared at COLING 2020.

Bio:

Daichi Mochihashi is an associate professor at the Institute of Statistical Mathematics, Japan. He obtained BS from The University of Tokyo and PhD from Nara Institute of Science and Technology in 1998 and 2005, respectively. He currently serves as an action editor at TACL and served as an area chair in some ACL conferences.

Technologies for Translation and Interpreting: Challenges and Latest Developments

Prof Ricardo Muñoz Martín, University of Bologna

16 April 2021

Title: Do translators dream of electric brains?

Abstract:

We cannot know whether Artificial Intelligence exists because we do not know yet what intelligence is. The way computers process natural languages is not really the way humans do it. Artificial neural networks have but a small, distant ressemblance with our brains’ biological structures. But, why should that matter? Researchers studied the flight of birds to develop the first planes. Birds and planes never were too similar and time has only separated them further. Biomimicry is inspiring, but evolutionary solutions are not necessarily the best for our machines. Furthermore, it leads to notions of competitiveness between humans and machines, not to symbiosis. A large portion of CAT research has focused on why people mistrust or dislike applications and systems. But, shouldn’t we be asking were did we go wrong? What do we know about the effects of digital tools on translators and their working ways? In order to develop practical applications with potential real-world use for translators, we need to approach the tasks in their natural(istic) environments, with a view on avoiding cognitive friction and to implement human in the loop testing that will ensure better pairings of humans and their digital tools.

Bio:

Ricardo Muñoz Martín is a (now seldom practising) freelance translator since 1987, ATA certified for English-Spanish in 1991. He studied at 6 European and American universities until 1993, when he was granted a PhD from UC Berkeley. Prof Muñoz lectured at 7 American and Spanish universities before he joined the Department of Interpreting & Translation of the University of Bologna, Italy. There, he directs the Laboratory for Multilectal Mediated Communication & Cognition (MC2 Lab), devoted to the empirical research of multilectal mediated communication events from the perspective of Cognitive Translatology—a theoretical frameowrk drawing from situated cognition. As a visiting scholar or guest speaker, Prof Muñoz has travelled widely in Europe, America and China. He is also a member of the TREC and HAL networks and co-editor of the journal Translation, Cognition & Behavior.

Technologies for Translation and Interpreting: Challenges and Latest Developments

Dr Konstantinos Chatzitheodorou, Strategic Agenda

Using technology in the translation quality assessment

19 March 2021

Abstract: The process of determining translation quality is subjective and relies on human judgments. Translation quality is affected by a variety of factors that are weighted differently in each translation task and can be viewed from different perspectives. Hence, it is not equally measurable or assessable (Almutairi, 2018). The talk will emphasize the importance of measuring translation quality and how it can be accomplished. The first part of the presentation will introduce different frameworks and software used in the process of translation evaluation focusing on error classification schemes available in both professional and academic word. The second part of the presentation will include a demonstration of Træval which is a Software-as-a-Service (SaaS) that allows humans to evaluate translation outputs. By providing an easy-to-use graphical interface, it assists researchers and users in this process. In particular, three different scenarios will be presented using the Dynamic Quality Framework – Multidimensional Quality Metrics (DQF-MQM) error typology (Lommel, 2018): an evaluation of a simple translation task, an evaluation task focused on the assessment of multi-word units, and, finally, a technology-aided evaluation task aiming to reduce subjectivity.

Dr Konstantinos Chatzitheodorou is a postdoctoral researcher at the Department of Foreign Languages, Translation and Interpreting, Ionian University. He received his PhD in Applied Translation Studies and Computational Linguistics from the Aristotle University of Thessaloniki. He holds a BA in Italian Language and Literature from the School of Italian Language and Literature, Aristotle University of Thessaloniki and an MSc in Informatics in Humanities from the Department of Informatics, Ionian University. He is also ECQA Certified Terminology Manager – Engineering. He is employed as a Computational Linguist in the private sector, assisting organizations to use language data to gain strategic insights. He has also worked as a Machine Translation Expert and Terminologist at the European Parliament – Directorate-General for Translation in Luxembourg. Over the years, Konstantinos has also contributed as a researcher to several EU projects in areas of his interest.

References

Almutairi, M.O.L., 2018. The objectivity of the two main academic approaches of translation quality assessment: arab spring presidential speeches as a case study (Doctoral dissertation, University of Leicester).

Chatzitheodorou, K. and Chatzistamatis, S, 2013. COSTA MT evaluation tool: An open toolkit for human machine translation evaluation. The Prague Bulletin of Mathematical Linguistics, 100(2013), pp.83-89.

Lommel, A., 2018. Metrics for translation quality assessment: a case for standardising error typologies. In Translation Quality Assessment (pp. 109-127). Springer, Cham.

Secară, A., 2005, March. Translation evaluation: A state of the art survey. In Proceedings of the eCoLoRe/MeLLANGE workshop, Leeds (Vol. 39, p. 44).