Isabelle Augenstein (University of Copenhagen). Understanding the Interplay between LLMs’ Utilisation of Parametric and Contextual Knowledge
[Abstract]
Language Models (LMs) acquire parametric knowledge from their training process, embedding it within their weights. The increasing scalability of LMs, however, poses significant challenges for understanding a model’s inner workings and further for updating or correcting this embedded knowledge without the significant cost of retraining. Moreover, when using these language models for knowledge-intensive language understanding tasks, LMs have to integrate relevant context, mitigating their inherent weaknesses, such as incomplete or outdated knowledge. Nevertheless, studies indicate that LMs often ignore the provided context as it can be in conflict with the pre-existing LM’s memory learned during pre-training. Conflicting knowledge can also already be present in the LM’s parameters, termed intra-memory conflict. This underscores the importance of understanding the interplay between how a language model uses its parametric knowledge and the retrieved contextual knowledge.
In this talk, I will aim to shed light on this important issue by presenting our research on evaluating the knowledge present in LMs, diagnostic tests that can reveal knowledge conflicts, as well as on understanding the characteristics of successfully used contextual knowledge.
Julia Kreutzer (Cohere Labs) Multilingual-First Language Modeling
[Abstract]
Large Language Models (LLMs) have traditionally been dominated by English-centric approaches, often overlooking the rich diversity of languages around the globe. In this talk, we explore the transformative potential of multilinguality across every stage of LLM development, from tokenization to inference and evaluation. We demonstrate how multilingual models not only enhance performance across diverse languages but also drive innovation. We will discuss how to prioritize multilinguality from the ground up, and to design evaluation to measure equitable performance across languages. By shifting the focus beyond English, we highlight how research in non-English languages can lead to breakthroughs that benefit the entire LLM community. This talk underscores the importance of embracing linguistic and cultural diversity as a catalyst for advancing LLMs and fostering global inclusivity in AI.
Timothée Lacroix (Mistral AI). TBA
[Abstract]
TBA
Roger K. Moore (University of Sheffield). Machines aren’t people – so are there limits on how we converse with them?
[Abstract]
Human beings have been talking with machines since the appearance of the first speech-based command-and-control systems in the 1980s. Over the intervening 40+ years, the capabilities of the core speech technology components have steadily improved such that high-accuracy automatic speech recognition, high-quality text-to-speech synthesis and LLM-powered spoken language dialogue are now readily available off-the-shelf. However, machines aren’t people; they have no lived experience of their own, their understanding is superficial rather than communicative, they struggle with continuous multimodal interactive engagement, and they present confusing affordances to potential interlocutors. This talk will discuss some of these challenges and ask whether they represent a limit on how we might converse with machines.
Carlos Ramisch (Aix Marseille University) and Manon Scholivet (LISN). Zen Research – Methodology in Natural Language Processing
[Abstract]
Experiments and empirical evaluation are at the core of NLP research. However, experimental design is often neglected: we do things without really understanding, leading to shaky conclusions, stressful paper writing, and unfavourable reviews. Our course focuses on scientific methodology: key notions, recommended practices and avoidable traps. As methodology is often considered boring, we focus on concrete examples and propose practical exercises to make the course more fun. Thus, we collaboratively build an ideal of scientific methodology, encouraging us to move our habits towards this ideal. First, we reflect on the scientific approach and experiments in NLP, discuss research questions and hypotheses, provide criteria to define them, and highlight their importance. Second, once we find our research question, we have to justify its potential impact and plausibility with respect to the literature. Third, the data we use, collect and annotate plays a crucial role to avoid regretting (hasty) experimental choices, so we discuss the creation and use of datasets. Fourth, scientific communication requires experience, but some tools can help: we study the structure of documents, presentations, tables and figures. We also cover reproducibility and open science.
Holger Schwenk (Meta). TBA
[Abstract]
TBA
François Yvon (CNRS, ISIR). Text Generation: Decoding with Constraints
[Abstract]
Text generation, contextual or non-contextual, is ubiquitous in the
current LLM era, as it serves as the most basic block in multiple
application contexts, from question answering and dialog systems to text
summarization and machine translation, and many more. Generation is thus
equally useful to compute deterministic and highly non-deterministic
mappings with various level of output constraints. Furthermore, text
generation is also used as a sub-routine of more complex generation
strategies, aiming to produce syntactically well-formed (e.g. for code
generation) or semantically consistent outputs, possibility through
multiple steps of generation (e.g, in chain-of-thoughts generation) or
to collect diverse samples from the generating distribution. In this
class, I will discuss generation algorithms with a focus on the
integration of various types of constraints in the decoder outputs.
To cover this considerable diversity of uses, multiple text generation
strategies have been proposed, some less well-known than others. In this
talk I will review various families of generation algorithms, from the
most basic ones to the more sophisticated approaches, so as to document,
as much as possible, the possible options that are available to text
generation users. The main focus for this year will be on decoding with
constraints.
Speakers from previous editions are here (2025) here (2024), here (2023), here (2022) and here (2021)