📄 Analyzing and Improving Cross-lingual Knowledge Transfer for Machine Translation

Summary

Most of the world's languages have little parallel data, so translating them depends on multilingual models reusing knowledge learned from better-resourced languages. This EAMT 2026 thesis abstract by David Stap (University of Amsterdam) studies that cross-lingual transfer from four angles.

1. It introduces Representational Transfer Potential (RTP), a measure based on how similar cross-attention context vectors are in multilingual NMT. It then adds an auxiliary loss that uses multi-parallel data to bring language representations closer together.
2. It adapts k-NN-MT to cross-lingual and multilingual datastores, using linear mappings to align representations across languages.
3. It examines how fine-tuning LLMs (7B–65B) on parallel data erodes skills such as formality control, and counters this by adding monolingual text to the training mix.
4. It varies language diversity across 132 translation directions and evaluates with BLEU and COMET.

The findings indicate that transfer happens at the representational level. Broader language coverage also helps unseen pairs, and mixed-data training preserves LLM abilities while improving quality. Together, these results give practical guidance for building translation for under-resourced languages.

Key points

• RTP correlates strongly with translation gains (Spearman's ρ = .77, p < 0.001). Multi-parallel overlap emerges as an under-explored predictor of transfer, alongside subword overlap and genetic distance.
• In k-NN-MT, datastores built from related high-resource languages beat bilingual ones for low-resource pairs. Language-group datastores match full multilingual ones while being up to 75% smaller and up to 5.3× faster.
• Parallel-only LLM fine-tuning harms formality steering, few-shot domain adaptation and document-level translation. The damage appears with as few as 18K examples and at every model scale tested.
• Mixed-data fine-tuning, combining parallel and monolingual text, keeps these capabilities and yields higher overall translation quality.
• Greater language diversity during fine-tuning improves both seen and unseen pairs and increases cross-lingual overlap in middle layers. The author releases the IdiomsInCtx-MT dataset, code and model checkpoints.