Elamur Research⌕
☰
home / papers / 2011 / A Joint Sequence Translation Model with Integrated Reordering

A Joint Sequence Translation Model with Integrated Reordering

Paper · 2011 · Nadir Durrani, Helmut Schmid and Alexander Fraser

Summary

Durrani, Schmid and Fraser propose an alternative to phrase-based statistical MT. It describes translation as one left-to-right chain of operations that builds source and target sentences together. The operations do two jobs. Some generate words: they emit minimal bilingual units (cepts), continue a discontinuous source cept, or emit a source word with no translation. Others reorder: they insert a gap, jump back to an open gap, or jump forward. An n-gram model over this operation chain scores translation and reordering jointly, so each kind of decision can condition on the other. The model is first stated as a generative story and then used inside a discriminative framework. The work builds on N-gram SMT, but it explores all reorderings during search rather than a small precomputed set. It also relies only on lexical cues, with no POS tags. On German-, Spanish- and French-to-English, the system beats Moses significantly for German and Spanish and matches it for French. This shows that small translation units combined with explicit reordering operations can rival phrase-based MT, including on long-range verb movement.

Key points

  • The model uses 4 translation operations and 3 reordering operations (Insert Gap, Jump Back (W), Jump Forward). English is produced strictly in order, while German words are placed alongside their translations.
  • The baselines are two Moses setups: one with no reordering limit and one with the default distortion limit of 6. Removing the limit costs Moses more than one BLEU point.
  • The best system has no hard reordering limit. It gains significantly over both Moses baselines (p < 0.05, pairwise bootstrap resampling with 1000 samples) for German- and Spanish-to-English.
  • Gappy source tuples did not improve BLEU because they inflate the set of translation options. For German, the 10-best options numbered 965,515 with gaps versus 256,992 without. The future cost estimates for these tuples were also unreliable.
  • Target-side discontinuities are removed from the training alignments by post-editing. The decoder and a binary of the corpus conversion tool were released.

Interesting ideas

  • I1. Reordering as explicit tokens in the sequence. Gaps and jumps become ordinary symbols that a sequence model can predict. Lexical context, such as an auxiliary verb, can then trigger a reordering, and short and long movements are handled by one mechanism. This framing could carry over to other monotone-generation tasks that need occasional reordering.
  • I2. Gaps as lexicalized placeholders. An open gap behaves like a nonterminal in hierarchical models, yet no grammar is required. A learned pattern like "hat … gelesen – read" therefore generalizes to unseen fillers.
  • I3. Minimal units with a unique segmentation. Phrase-based MT lets many segmentations of the same sentence pair compete. Small units with exactly one segmentation per sentence pair avoid this spurious ambiguity. Frequent phrases can still be memorized as likely operation sequences.
  • I4. Pruning composite units by a cost check. A gappy tuple is discarded if its estimated cost is lower than the sum of its parts' best scores. This shrinks the search space and recovered most of the lost quality. A similar test could prune combinatorial candidate sets elsewhere.
  • I5. Inspecting the search, not just the score. When gappy units failed to help, the authors counted the translation options and examined future-cost errors to locate the cause. This is a useful habit whenever a richer representation does not improve results.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of StuttgartStuttgart · Germany75 works · 46,113 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Machine translation · 958 works

Cite

Cite as Nadir Durrani, Helmut Schmid and Alexander Fraser. 2011. A Joint Sequence Translation Model with Integrated Reordering. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{durrani2011joint,
  title = {{A Joint Sequence Translation Model with Integrated Reordering}},
  author = {Nadir Durrani, Helmut Schmid and Alexander Fraser},
  booktitle = {Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies},
  year = {2011},
  url = {https://aclanthology.org/P11-1105/}
}

References

22 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and R. L. Mercer. 1993. The mathematics of statistical machine translation: parameter estimation. Computational Linguistics, 19(2):263–311. find
  2. David Chiang. 2007. Hierarchical phrase-based translation. Computational Linguistics, 33(2):201–228.
  3. Josep Maria Crego and Franois Yvon. 2009. Gappy translation units under left-to-right smt decoding. In Proceedings of the meeting of the European Association for Machine Translation (EAMT), pages 66–73, Barcelona, Spain. find
  4. Josep Maria Crego and François Yvon. 2010. Improving reordering with linguistically informed bilingual n-grams. In Coling 2010: Posters, pages 197–205, Beijing, China, August. find
  5. Coling 2010 Organizing Committee. find
  6. Josep M. Crego, Marta R. Costa-juss, Jos B. Mario, and Jos A. R. Fonollosa. 2005a. Ngram-based versus phrasebased statistical machine translation. In In Proceedings of the International Workshop on Spoken Language Technology (IWSLT05, pages 177–184. find
  7. Josep M. Crego, José B. Marin̂o, and Adrià de Gispert. 2005b. Reordered search and unfolding tuples for ngram-based SMT. In Proceedings of the 10th Machine Translation Summit (MT Summit X), pages 283– 289, Phuket, Thailand. find
  8. Michel Galley and Christopher D. Manning. 2010. Accurate non-hierarchical phrase-based translation. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 966– 974, Los Angeles, California, June. Association fo… find
  9. Philipp Koehn and Barry Haddow. 2009. Edinburgh’s submission to all tracks of the WMT 2009 shared task with reordering and speed improvements to Moses. In Proceedings of the Fourth Workshop on Statistical Machine Translation, pages 160–164, Athens, Greece, March. Association for Computational Lingui… find
  10. Philipp Koehn, Franz J. Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Proceedings of the Human Language Technology and North American Association for Computational Linguistics Conference, pages 127–133, Edmonton, Canada. find
  11. Philipp Koehn, Amittai Axelrod, Alexandra Birch Mayne, Chris Callison-Burch, Miles Osborne, and David Talbot. 2005. Edinburgh system description for the 2005 iwslt speech translation evaluation. In International Workshop on Spoken Language Translation 2005.
  12. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Proc… find
  13. Philipp Koehn. 2004a. Pharaoh: A beam search decoder for phrase-based statistical machine translation models. In AMTA, pages 115–124. find
  14. Philipp Koehn. 2004b. Statistical significance tests for machine translation evaluation. In Dekang Lin and Dekai Wu, editors, Proceedings of EMNLP 2004, pages 388–395, Barcelona, Spain, July. Association for Computational Linguistics.
  15. Zhifei Li, Chris Callison-burch, Chris Dyer, Juri Ganitkevitch, Sanjeev Khudanpur, Lane Schwartz, Wren N. G. Thornton, Jonathan Weese, and Omar F. Zaidan. 2009. Joshua: An open source toolkit for parsing-based machine translation.
  16. J.B. Mariño, R.E. Banchs, J.M. Crego, A. de Gispert, P. Lambert, J.A.R. Fonollosa, and M.R. Costa-jussà. 2006. N-gram-based machine translation. Computational Linguistics, 32(4):527–549.
  17. I. Dan Melamed. 2004. Statistical machine translation by parsing. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, Barcelona, Spain. find
  18. Franz J. Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19–51. find
  19. Franz J. Och and Hermann Ney. 2004. The alignment template approach to statistical machine translation. Computational Linguistics, 30(1):417–449. find
  20. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL ’02, pages 311–318, Morristown, NJ, USA. Association for Computational Lingui…
  21. Andreas Stolcke. 2002. SRILM - an extensible language modeling toolkit. In Intl. Conf. Spoken Language Processing, Denver, Colorado. find
  22. Omar F. Zaidan. 2009. Z-MERT: A fully configurable open source tool for minimum error rate training of machine translation systems. The Prague Bulletin of Mathematical Linguistics, 91:79–88. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/durrani-2011-joint-sequence-translation-model-integrated/