Elamur Research⌕
☰
home / papers / 2026 / A Longitudinal Study of the Adoption of Specialized MT Systems in Canadian Parliamentary Translation

A Longitudinal Study of the Adoption of Specialized MT Systems in Canadian Parliamentary Translation

Summary

Most studies of how professional translators adopt machine translation capture a single moment. This paper instead follows adoption as it unfolds. Since 2023, the Translation Bureau's parliamentary service has been receiving Hawkeye from Canada's National Research Council. These are Sockeye-based NMT systems for French–English, trained only on Canadian parliamentary data and built into Prism, the Parliament's publishing workflow tool. Using Hawkeye is optional: translators must actively insert its output.

The authors analyse anonymized Prism logs covering roughly 2.5 years. Each record stores the source chunk, the Hawkeye output, the translator's draft and, where one exists, the reviser's version. With these logs they track usage month by month, profile individual translators' habits, and estimate effects on translations using BLEU (as a similarity measure) and TER (as a measure of revision effort).

Adoption rose sharply and then levelled off. Translators tended to settle into consistent habits, either using the system routinely or avoiding it. Drafts made with Hawkeye stayed closer to the MT output and needed somewhat less revision. This offers rare real-world, long-term evidence on optional MT adoption in a high-stakes setting.

Key points

  • Dataset: 286,858 records (about 25.0M words, roughly 87 words per chunk) collected from 15 May 2023 to 12 December 2025. The logs cover House of Commons Debates and Committees, and 69.3% of chunks are English→French.
  • Hawkeye training and maintenance:
    • The training set has 26.8M segments, about 400M+ words per language.
    • A shared base model is fine-tuned for each domain.
    • The systems handle XML-structured input.
    • They are rebuilt from scratch three times a year.
  • Adoption over time: Hawkeye was used for 46.9% of chunks overall. Usage grew from 16% in May 2023 to about 55% by fall 2024 and stayed at roughly that level in 2025. In 2025, use was higher into French (57.8%) than into English (50.0%).
  • Translator profiles (July–December 2025): 44% were non-users, 41% regular users, and only about 15% occasional users.
  • Revision effort: Revisers' average TER was 5.89 for Hawkeye-assisted drafts versus 7.22 for others, a difference of 1.33 ± 0.16. This gap largely disappeared in 2025. Use of other MT tools, such as DeepL Pro and GCtranslate, cannot be detected in the logs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Topics: Machine translation

Cited by

2 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 1 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{simard2026longitudinal,
  title = {{A Longitudinal Study of the Adoption of Specialized MT Systems in Canadian Parliamentary Translation}},
  author = {Simard, Michel and Leal-Wyss, Jeniffer and Bernier-Colborne, Gabriel and Knowles, Rebecca},
  booktitle = {Proceedings of the 26th Annual Conference of the European Association for Machine Translation (Volume 2)},
  year = {2026},
  url = {https://aclanthology.org/2026.eamt-2.32/}
}

References

9 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. O’Brien, Audrey. 2002. Prism: The house of commons integrated technology project. Canadian Parliamentary Review, 25(2):20–22. find
  2. Cadwell, Patrick, Sharon O’Brien, and Carlos Teixeira. 2018. Resistance and accommodation: factors for the (non-) adoption of machine translation among professional translators. Perspectives, 26(3):301– 321. find
  3. O’Brien, Sharon. 2024. Human-centered augmented translation: against antagonistic dualisms. Perspectives, 32(3):391–406. do Carmo, Félix and Joss Moorkens. 2022. Translation’s new high-tech clothes. In Huertas-Barros, Elsa, Gary Massey, and David Katan, editors, The human translator in the 2020s, p… find
  4. Guerberof Arenas, Ana. 2025. Perspectives on machine translation, post-editing, and automation. In Walker, Callum and Joseph Lambert, editors, The Routledge handbook of the translation industry, pages 186–204. Routledge. find
  5. Hieber, Felix, Michael Denkowski, Tobias Domhan, Barbara Darques Barros, Celina Dong Ye, Xing Niu, Cuong Hoang, Ke Tran, Benjamin Hsu, Maria Nadejde, Surafel Lakew, Prashant Mathur, Anna Currey, and Marcello Federico. 2022. Sockeye 3: Fast neural machine translation with pytorch. find
  6. Knowles, Rebecca, Samuel Larkin, Marc Tessier, and Michel Simard. 2023. Terminology in neural machine translation: A case study of the Canadian Hansard. In Nurminen, Mary, Judith Brenner, Maarit Koponen, Sirkku Latomaa, Mikhail Mikhailov, Frederike Schierl, Tharindu Ranasinghe, Eva Vanmassenhove, Se… find
  7. Knowles, Rebecca, Samuel Larkin, Michel Simard, Marc A Tessier, Gabriel Bernier-Colborne, Cyril Goutte, and Chi-kiu Lo. 2024. Some tradeoffs in continual learning for parliamentary neural machine translation systems. In Knowles, Rebecca, Akiko Eriguchi, and Shivali Goel, editors, Proceedings of the… find
  8. Macken, Lieve, Daniel Prou, and Arda Tezcan. 2020. Quantifying the effect of machine translation in a Post, Matt. 2018. A call for clarity in reporting BLEU scores. In Bojar, Ondřej, Rajen Chatterjee, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes… find our card
  9. Zaretskaya, Anna. 2015. The use of machine translation among professional translators. In Proceedings of the EXPERT Scientific and Technological Workshop, pages 1–12. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/simard-2026-longitudinal-study-adoption-specialized-mt/