From Insights to Impact: Actionable Interpretability for Neural Machine Translation

Summary

This EAMT 2026 thesis abstract tackles a persistent disconnect: interpretability research reveals much about how language models work internally, yet those findings seldom benefit the people who actually use machine translation. The dissertation builds and tests interpretability methods end to end, aiming to make MT more trustworthy and steerable. Part I offers tools for tracing context usage: the Inseq attribution library, the PECoRe framework that spots context-sensitive output tokens and links them to the contextual cues behind them, and MIRAGE, which derives faithful citations for retrieval-augmented answers from model internals. Part II addresses personalization, via the retrieval-based prompting method RAMP for formality and gender control, and via sparse autoencoder latents that steer literary translations toward an individual translator's style about as well as few-shot prompting, without long prompts. Part III moves to real workflows with user studies (DivEMT; QE4PE, where error highlights cut critical errors by 15–20%) and a broad benchmark of unsupervised quality estimation. The work matters because it measures interpretability by its effect on professionals' productivity and output quality, offering a template for transparent translation tools.

Key points

DivEMT involved 18 professional translators and six typologically diverse languages. MT-driven productivity gains depended strongly on the language pair: related languages gained a lot and distant ones little. BLEU and COMET were weak predictors of these gains.
QE4PE involved 42 professional post-editors in two language directions. The study compared highlights from supervised QE, unsupervised uncertainty-based methods and oracle human annotations. Coarse metrics showed no clear differences between highlight sources. However, every highlighted condition reduced critical errors by 15–20% compared with plain post-editing, at the cost of more cognitive effort.
Unsupervised QE benchmark covered 12 translation directions. The variance of token log-probabilities under Monte Carlo Dropout matched supervised state-of-the-art methods. With calibration, it approached near human inter-annotator agreement. Annotator subjectivity noticeably shifted metric rankings.
PECoRe uncovered gender-agreement errors in multilingual context-aware MT, caused by faulty anaphora resolution. It also showed occasional overreliance on target-side context. MIRAGE citations matched or beat self-citation approaches.
Venues for the components: Inseq (ACL Demo 2023, XAI Demo 2024), PECoRe (ICLR 2024), MIRAGE (EMNLP 2024), RAMP (ACL 2023), SAE-based style steering (EACL 2026), QE4PE (TACL 2025). Funding came from NWO's InDeep project.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Topics: Machine translation

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{sarti2026insights,
  title = {{From Insights to Impact: Actionable Interpretability for Neural Machine Translation}},
  author = {Sarti, Gabriele},
  booktitle = {Proceedings of the 26th Annual Conference of the European Association for Machine Translation (Volume 1)},
  year = {2026},
  url = {https://aclanthology.org/2026.eamt-1.1/}
}

References

9 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

Qi, Jirui, Gabriele Sarti, Raquel Fernández, and Arianna Bisazza. 2024. Model internals-based answer attribution for trustworthy retrieval-augmented generation. In Conference on Empirical Methods in NLP (EMNLP). find
Sarti, Gabriele, Arianna Bisazza, Ana Guerberof-Arenas, and Antonio Toral. 2022. DivEMT: Neural machine translation post-editing effort across typologically diverse languages. In Conference on Empirical Methods in NLP (EMNLP). find
Sarti, Gabriele, Nils Feldhus, Ludwig Sickert, and Oskar van der Wal. 2023a. Inseq: An interpretability toolkit for sequence generation models. In Proceedings of the Association for Computational Linguistics (ACL): Demos. find
Sarti, Gabriele, Phu Mon Htut, Xing Niu, Benjamin Hsu, Anna Currey, Georgiana Dinu, and Maria Nadejde. 2023b. RAMP: Retrieval and attribute-marking enhanced prompting for attribute-controlled translation. In Proceedings of the Association for Computational Linguistics (ACL). find
Sarti, Gabriele, Grzegorz Chrupała, Malvina Nissim, and Arianna Bisazza. 2024a. Quantifying the plausibility of context reliance in neural machine translation. In 12th International Conference on Learning Representations (ICLR), May. find
Sarti, Gabriele, Nils Feldhus, Jirui Qi, Malvina Nissim, and Arianna Bisazza. 2024b. Democratizing advanced attribution analyses of generative language models with the inseq toolkit. In 2nd World Conference on XAI: Demos. CEUR.org. find
Sarti, Gabriele, Vilém Zouhar, Grzegorz Chrupała, Ana Guerberof-Arenas, Malvina Nissim, and Arianna Bisazza. 2025a. QE4PE: Word-level quality estimation for human post-editing. Transactions of the Association of Computational Linguistics (TACL). find
Sarti, Gabriele, Vilém Zouhar, Malvina Nissim, and Arianna Bisazza. 2025b. Unsupervised word-level quality estimation for machine translation through the lens of annotators (dis)agreement. Conference on Empirical Methods in NLP (EMNLP). find
Scalena, Daniel, Gabriele Sarti, Arianna Bisazza, Elisabetta Fersini, and Malvina Nissim. 2026. Steering large language models for machine translation personalization. Proceedings of the European Chapter of the Association for Computational Linguistics (EACL). find

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sarti-2026-insights-impact-actionable-interpretability-neural/