EU Institutional LLM Multilingual Continual Pretraining Paper Wins EuroHPC Best Paper Award

A paper titled "Mitigating Catastrophic Forgetting in Multilingual Continual Pretraining: Lessons from EU Institutional LLMs" has won Best Paper at the EuroHPC User Days, organized by the European High Performance Computing Joint Undertaking. The paper was drafted by colleagues from DG Translation, the Directorate-General for Communications Networks, Content and Technology, and the Joint Research Centre of the European Commission, and was evaluated by a peer review committee. The research tested the EU Institutional LLM's general reasoning, language understanding, and toxicity resistance across all 24 EU official languages, finding that catastrophic forgetting could be reduced but not fully avoided through a combination of learning rate, data, and training process.

The paper offers reusable practical methods for building high-quality multilingual LLMs suited to European public-sector needs, with conclusions directly serving AI translation and language tools available to EU public administrations, small businesses, academia, and non-governmental organisations. For education, improvements in multilingual model capability boundaries and public-sector language infrastructure can affect the availability of teaching resources generated in EU official languages, cross-language learning tools, and translation-based education products. The evidence is limited in that the paper states catastrophic forgetting can be reduced but not fully avoided, and the toxicity resistance conclusion is limited to the 24 official languages tested. AI translation and language tools available to EU public administrations, small businesses, academia, and non-governmental organisations may benefit from more stable multilingual model capabilities. Cross-language teaching resources and translation-based education products relying on EU official languages may see changes in underlying model language coverage and stability as the research lessons are applied.