Elamur Research⌕
☰
home / papers / 2010 / Convergence of Translation Memory and Statistical Machine Translation

Convergence of Translation Memory and Statistical Machine Translation

Summary

Translation memories (TM) and statistical machine translation (SMT) grew up in separate communities, even though translation agencies increasingly want to use both. Koehn and Senellart propose combining them. They retrieve the closest fuzzy match from the TM and keep the parts of its translation that still apply. The SMT system then translates only the source words that differ.

The first method wraps the trusted target fragments in Moses XML markup, so the decoder only fills the gaps. The second recasts the same match as a very large hierarchical phrase rule (VLR) with non-terminals at the mismatch positions. These rules go into a separate rule table of a hierarchical Moses model.

Experiments used a commercial product-manual corpus and the English–French JRC-Acquis corpus. The XML approach beats both TM and SMT baselines for matches of roughly 80–99%. The VLR approach beats every other method across all tested ranges from 70% upward. The authors conclude that fuzzy matches are best treated as one more source of translation rules inside SMT, rather than as a standalone tool.

Key points

  • Fuzzy match score: defined as one minus the word-level edit distance, divided by the length of the longer segment. Character-level edit distance breaks ties.
  • Locating the mismatch: edit distance finds the differing words on the source side. Word alignment from GIZA++ (IBM Models plus symmetrization) maps them onto the TM target. This alignment comes for free when the Moses pipeline is trained on the TM data.
  • Data:
    • Acquis: about 1.17M segments (≈24M English words), with a 4,107-segment test set reused from Koehn et al. (2009).
    • Product: about 83K segments (≈1M English words), 2,000 test segments, and short segments averaging 12 words.
  • Results: XML frames win over SMT and TM only for high fuzzy matches. VLR, which combines all best-scoring TM matches, all their target versions, and jointly tuned feature weights, wins everywhere at ≥70%. This includes the 70–79% band, where the XML method falls behind SMT.
  • Estimated practical gain: on the 70–75% and 75–80% subsets, the method's BLEU matches what raw TM achieves about 10 points higher in fuzzy match score. The authors read this as roughly a 5–10% fuzzy-match-equivalent improvement. No user study was carried out.

Interesting ideas

  • I1.

    Constrained decoding by subtraction: keep the retrieved translation as fixed output and leave only the changed source words for the decoder. This idea could carry over to any retrieve-and-edit setup, such as updating legal or technical documents with small revisions.

  • I2.

    A short list of fallback rules for messy alignments:

    • insert every mismatched source word;
    • remove target words aligned to mismatched TM words;
    • when alignment is missing, use the alignment of the preceding word.

    Simple heuristics like these make edit-transfer systems robust without needing syntax.

  • I3.

    TM matches as grammar rules: turning a whole TM match into one big hierarchical rule with gaps lets memory retrieval and statistical scoring be tuned jointly. This avoids a hard choice between "use the memory" and "use MT."

  • I4.

    Evaluate by fuzzy-match band: splitting the test set by fuzzy match range shows where each method helps. Expressing BLEU gains as "equivalent to a higher fuzzy match" makes the results meaningful to translation agencies.

  • I5.

    Highlight machine-filled spans: marking which parts of the output came from SMT tells post-editors where to look for errors. This is a cheap interface addition for any hybrid human–MT workflow.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Topics: Machine translation

Cited by

62 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 61 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{koehn2010convergence,
  title = {{Convergence of Translation Memory and Statistical Machine Translation}},
  author = {Koehn, Philipp and Senellart, Jean},
  booktitle = {Proceedings of the Second Joint EM+/CNGL Workshop: Bringing MT to the User: Research on Integrating MT in the Translation Industry},
  year = {2010},
  url = {https://aclanthology.org/2010.jec-1.4/}
}

References

19 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Biçici, Ergun and Marc Dymetman. 2008. Dynamic Translation Memory: Using Statistical Machine Translation to Improve Translation Memory. In Gelbukh, Alexander F., editor, Proceedings of the 9th Internation Conference on Intelligent Text Processing and Computational Linguistics (CICLing), volume 4919… find
  2. Brown, Peter F., Stephen A. Della-Pietra, Vincent J. Della-Pietra, and Robert L. Mercer. 1993. The Mathematics of Statistical Machine Translation. Computational Linguistics, 19(2):263–313. find
  3. Chiang, David. 2007. Hierarchical Phrase-Based Translation. Computational Linguistics, 33(2):201–228. find our card
  4. Dugast, Loı̈c, Jean Senellart, and Philipp Koehn. 2007. Statistical Post-Editing on SYSTRAN’s Rule-Based Translation System. In Proceedings of the Second Workshop on Statistical Machine Translation, pages 220–223, Prague, Czech Republic, June. Association for Computational Linguistics. find
  5. He, Yifan, Yanjun Ma, Andy Way, and Josef van Genabith. 2010. Integrating N-best SMT Outputs into a TM System. In Coling 2010: Posters, pages 374–382, Beijing, China, August. find
  6. Coling 2010 Organizing Committee. find
  7. Hoang, Hieu, Philipp Koehn, and Adam Lopez. 2009. A unified framework for phrase-based, hierarchical, and syntax-based statistical machine translation. In Proceedings of the International Workshop on Spoken Language Translation (IWSLT), pages 152–159, December. find
  8. Koehn, Philipp, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Christopher J. Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open Source Toolkit for Statistical Machine Translati… find
  9. Koehn, Philipp, Alexandra Birch, and Ralf Steinberger. 2009. 462 Machine Translation Systems for Europe. In Proceedings of the Twelfth Machine Translation Summit (MT Summit XII). International Association for Machine Translation. find
  10. Koehn, Philipp. 2010. Statistical Machine Translation. Cambridge University Press. find
  11. Lopez, Adam. 2007. Hierarchical Phrase-Based Translation with Suffix Arrays. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 976–985. find
  12. Och, Franz Josef, Christoph Tillmann, and Hermann Ney. 1999. Improved Alignment Models for Statistical Machine Translation. In Proceedings of the Joint Conference of Empirical Methods in Natural Language Processing and Very Large Corpora (EMNLP-VLC), pages 20–28. find our card
  13. Simard, Michel and Pierre Isabelle. 2009. Phrase-based Machine Translation in a Computer-assisted Translation Environment. In Proceedings of the Twelfth Machine Translation Summit (MT Summit XII). International Association for Machine Translation. find
  14. Smith, James and Stephen Clark. 2009. EBMT for SMT: A New EBMT-SMT Hybrid. In Forcada, Mikel L. and Andy Way, editors, Proceedings of the 3rd International Workshop on Example-Based Machine Translation, pages 3–10. find
  15. Soricut, Radu and Abdessamad Echihabi. 2010. TrustRank: Inducing Trust in Automatic Translations via Ranking. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 612–621, Uppsala, Sweden, July. Association for Computational Linguistics. find
  16. Specia, Lucia, Marco Turqui, Zhuoran Wang, John Shawe-Taylor, and Craig Saunders. 2009. Improving the Confidence of Machine Translation Quality Estimates. In Proceedings of the Twelfth Machine Translation Summit (MT Summit XII). International Association for Machine Translation. find
  17. Steinberger, Ralf, Bruno Pouliquen, Anna Widiger, Camelia Ignat, Tomaz Erjavec, Dan Tufis, and Daniel Varga. 2006. The JRC-Acquis: A multilingual aligned parallel corpus with 20+ languages. In 5th Edition of the International Conference on Language Resources and Evaluation (LREC ’06), pages 2142–214… find our card
  18. Zhechev, Ventsislav and Josef van Genabith. 2010. Seeding Statistical Machine Translation with Translation Memory Output through Tree-Based Structural Alignment. In Proceedings of the 4th Workshop on Syntax and Structure in Statistical Translation, pages 43–51, Beijing, China, August. find
  19. Coling 2010 Organizing Committee. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/koehn-2010-convergence-translation-memory-statistical-machine/