Machine Learning/Deep Learning

Dr Iacer Calixto, University of Amsterdam

15 June 2021

Wikipedia Entities as Rendezvous across Languages: Grounding Multilingual Language Models by Predicting Wikipedia Hyperlinks

Abstract: Masked language models have quickly become the de facto standard when processing text. Recently, several approaches have been proposed to further enrich word representations with external knowledge sources such as knowledge graphs. However, these models are devised and evaluated in a monolingual setting only. In this work, we propose a language-independent entity prediction task as an intermediate training procedure to ground word representations on entity semantics and bridge the gap across different languages by means of a shared vocabulary of entities. We show that our approach effectively injects new lexical-semantic knowledge into neural models, improving their performance on different semantic tasks in the zero-shot cross-lingual setting. As an additional advantage, our intermediate training does not require any supplementary input, allowing our models to be applied to new datasets right away. In our experiments, we use Wikipedia articles in up to 100 languages and already observe consistent gains compared to strong baselines when predicting entities using only the English Wikipedia. Further adding extra languages lead to improvements in most tasks up to a certain point, but overall we found it non-trivial to scale improvements in model transferability by training on ever increasing amounts of Wikipedia languages.

Bio: Iacer Calixto is a Marie-Sklodowska Curie Global Fellow at the University of Amsterdam and a visiting postdoctoral fellow in the Center for Data Science in New York University. He is interested in modelling human language production and understanding in a broad sense. In so doing, his research focuses on meaning as a function of the interaction between language, visual perception, commonsense, and structured knowledge. Some of the concrete questions he has been working on include how general-purpose (vision and) language models generalise, and also how to make use of Wikipedia entities to improve neural language models (the topic of our talk).

Natural Language Processing

Dr Sheila Castilho, ADAPT Centre

14 June 2021

Title: Towards Document-Level Human MT Evaluation: what have we learned so far?


Document-level human evaluation of Machine Translation (MT) has been raising interest in the community. However, little is known about the issues of using document-level methodologies to assess MT quality. This presentation will explore what has been done so far in document-level human MT evaluation, touching the issues of inter-annotator agreement, effort and misevaluation.


Sheila Castilho graduated in Linguistics. She holds a joint Master in Natural Language Processing from the University of Wolverhampton –UK and the University of Algarve – PT, and currently, she is an Irish Research Council Research Fellow at the Adapt Centre, doing her research on Machine Translation evaluation in the school of computing, in DCU. She has authored several journal articles and book chapters on translation technology, post-editing of machine translation, user evaluation of machine translation, and translators’ perception of machine translation. Her research interests include machine translation, post-editing, machine and human translation evaluation, document-level machine translation, usability, and translation technologies.

Technologies for Translation and Interpreting: Challenges and Latest Developments

Dr Claudio Fantinuoli, University of Mainz/Germersheim and KUDO Way

11 June 2021

Title: Making sense of AI in the interpreter workstation

Speaker’s short bio: Dr. Claudio Fantinuoli is researcher and lecturer at the University of Mainz and Head of Innovation at KUDO Inc. He researches in the area of Natural Language Processing applied to human and machine interpreting. He lectures Language Technologies and Conference Interpreting at the University of Mainz and at the Postgraduate Center of the University of Vienna. He is the founder of InterpretBank.

Abstract of the lecture: Interpretation is at the verge of a third technical revolution. This will bring about a deeper integration of technology in the interpreter workstation. Artificial Intelligence is becoming integral part of computer-assisted interpreting (CAI) tools and is now allowing machine learning to enter the workflow of professional interpreters. CAI tools can create ad-hoc linguistic resources, suggest in real-time translations, numbers and proper names and automatize several aspects around the service provision. In this presentation I will discuss the promise of AI for the interpreting profession, its potentials and risks.

Digital Humanities

Prof. Umar Ryad, KU Leuven, Belgium

10 June 2021

Title: Computational Analysis of “Al-Manar” (Lighthouse)


The talk should not be seen as part of the ongoing debate about the efficiency of distant reading vis-à-vis close reading as such. The study is rather a test case of how far computational sciences and Islamic and Arabic studies can be “partners” in answering cultural-religious and historical questions related to the Arab and Muslim world. The research, which has been conducted in collaboration with Emad Mohamed of Wolverhampton,  mainly discusses methods that could help us computationally track, quantify, and explain the development of religious concerns of reform as reflected in the well-known Muslim reformist journal al-Manār (Lighthouse), published by the Muslim reformer Muhammad Rashīd Riḍā (1865-1935) from 1898 until 1935 in Cairo. We employ quantitative and qualitative methods based on al-Manār-corpus by using morphological processing, topic modeling in order to examine the thematic co-occurrences of the topics and lexemes pertaining to Muslim thinking and societies in Riḍā’s time. As we shall see, this distant digital reading will be supported by qualitative historical close analysis to map these topics in relation to the events that triggered them. In the present case study, by looking for disciplinary connections we shall test computational quantitative models against the background of historical qualitative evidence. We seek for overarching narratives by using a combination of quantitative data of frequent topics as they appeared in al-Manār and interpret them by using qualitative micro histories that illustrate these topics on a different level. We shall see how the results of digital history can be evaluated against the traditional interpretative reading of historical sources by using al-Manār as a case study. By this present collaborative endeavor, we therefore aim to bring statistics and algorithms closer to human historical interpretations.


Umar Ryad is Professor of Arabic and Islamic Studies at the University of Leuven, member of the Young Academy of Belgium (2018-2023) and holder of Alexander von Humboldt Fellowship for Experienced Researchers (2021-2024) at the Centrum für Nah- und Mittelost-Studien (CNMS), Philipps-Universität Marburg. He is also currently the chair of the research unit East Asian and Arabic Studies, and the director of Leuven Center for the Study of Islam, Culture and Society (LCSICS). Prior he has worked as assistant professor at the University of Leiden (2008-2014) and as associate professor at Utrecht University (2014-2017). He earned a BA in Islamic Studies in English from Al-Azhar University in Cairo (1998), followed by an MA degree in Islamic Studies (2001, Cum Laude) and a PhD degree (2008), both from Leiden University. His current research also includes the dynamics of the networks of pan-Islamist movements, Arab reception of Orientalism, Muslim polemics on Christianity, the European trans-imperial connections with the Hajj, transnational Islam in the modern world and the application of Digital Humanities to Arabic and Islamic Studies. He led a European Research Council (ERC) project which focused on the “History of Muslims in Interwar Europe and European transcultural history” (2014-2019). The project studied the intellectual and religio-political roles played by Muslim “intellectual agents” during the interwar years and up until the end of World War II (1918-1946). He is also a co-applicant of two ongoing international research projects: 1) Marie Curie ITN-project “Mediating Islam in the Digital Age” (MIDA) and 2) research consortium “The Computational Study of Culture: Cultural Analytics for Modern Arab and Muslim Studies”, which is funded by Qatar National Research Fund and is based at Doha Institute for Graduate Studies.

Natural Language Processing

Dr Félix do Carmo, University of Surrey

7 June 2021

Title: Translation & technology: tension and interaction


Translation and technology have a tense relationship. This does not arise from a conflict between the creativity in language and the efficiency of technology. In fact, translation has always looked for efficiency, at least since it became a profession. The tension comes from lack of clarity on which of them takes the wheel and how they can best interact. Yes, just like an old couple. In this talk, I will present a translator’s perspective on translation technology. This will start from computer-assisted translation and terminology management, technologies that explore the best methods for structuring information. I will take a peek at the history of MT, to quickly get to a discussion on what MT is today, and why it should not be seen as the technology that drives the translation process. I will cover the power of data-driven approaches and current neural methods. But I will also mention how MT and NLP often conceptualise translation as a straightforward process of transfer between stable linguistic systems, resulting in an impoverished approach to one of the most complex phenomena created by humanity. The richest part of the presentation will be on the interaction between technology and translation. In it, I will talk about interactive machine translation and interactive post-editing, and I will refer to quality estimation of machine translation and automatic post-editing. In the end, I hope to incentivise you to think about translation as a fascinatingly complex process which we will need to study for many years to come, with the help of the powerful technologies at our reach.


Félix do Carmo is a Senior Lecturer in Translation and Natural Language Processing at the Centre for Translation Studies of the University of Surrey, in the United Kingdom. He finished his PhD at the University of Porto, where he was a Guest Lecturer, after a career of more than twenty years as a translator and a translation company owner in Portugal. He was then granted a prestigious two-year EDGE-MSCA fellowship to work as a post-doctoral researcher in Dublin City University, Ireland. He has presented his work in international conferences and published in international publications, such as the recent article in “Translation Spaces” about marks of time and money in translation, and another one in the “Machine Translation” journal about automatic post-editing. His research interests cover the translation process and translation technologies, besides workflows and ethical issues in professional translation.

Machine Learning/Deep Learning

Professor Ryan Cotterell, ETH Zurich

25 May 2021

Title: Two New Insights into Beam Search


As a simple search heuristic, beam search has been used to decode models developed by the NLP community for decades. Indeed, it is noteworthy that beam search is one of the few NLP algorithms that has stood the test of time: It has remained a cornerstone of NLP systems since the 1970s (Reddy, 1977). As such, beam search became the natural choice for decoding neural probabilistic text generators—whose design makes evaluating the full search space impossible While there is no formal guarantee that beam search will return—or even approximate—the highest-scoring candidate under a model, it has repeatedly proven its merit in practice and, thus, has largely been tolerated—even embraced—as NLP’s go-to search heuristic.

This talk further embraces beam search. We discuss two novel formal insights into beam search. In the first act, we discuss an algorithmic advance that allows beam search to be prioritized, i.e. it returns the best hypothesis (modulo the beam size) first. Our algorithmic extension yields a Dijkstra-ified beam search that provably emulates standard beam search. In the second act, we draw a connection between the uniform information density hypothesis from cognitive science and beam search’s efficacy as a search heuristic. We offer a linguistic reason why beam search may work so well in practice even though, as an approximation to the argmax, it may be arbitrarily bad. The work described in this talk is described in publications at TACL (20200 and EMNLP (2020) and won an honorable mention for best paper at the latter.


Ryan Cotterell is an Assistant Professor of computer science at ETH Zurich. He is also affiliated with the Computer Laboratory (Department of Computer Science and Technology) at the University of Cambridge where he was an Assistant Professor (Lecturer in the UK system) from 2018 to 2020. His research papers have received best-paper awards at ACL 2017 and EACL 2017 and his papers were runners-up for best paper at NAACL 2016, EMNLP 2016, ACL 2019, EMNLP 2020, and EACL 2021. During his Ph.D., he was awarded fellowships from Facebook, Fulbright, DAAD, and NDSEG. He was also awarded the Jelinek fellowship at Johns Hopkins. At ETH Zurich, his group is jocularly called Rycolab; its web presence may be found here