Elamur Research⌕
☰
home / papers / 2014 / Anaphora Models and Reordering for Phrase-Based SMT

Anaphora Models and Reordering for Phrase-Based SMT

Summary

This WMT14 system description from Uppsala University explores how discourse-level and word-order phenomena can be modelled in phrase-based SMT. Most of the work builds on Docent, a document-level decoder. Instead of beam search, Docent uses stochastic hill climbing over complete document translations, so feature functions can inspect the whole translated text.

For English–French, the authors add two neural components. The first is a pronominal anaphora model that picks the French pronoun agreeing with a possibly distant antecedent. The second is a dependency projection model that scores how source-side dependency arcs map onto target words. Unfortunately, the submitted anaphora model contained a serious bug, so its results are not analysed; only the baseline score is reported.

For English–German, the focus is reordering of verbs and particles. The authors test rule-based post-ordering, applying TreeTagger POS patterns to put German into English-like order. They also test tunable POS-based phrase-distortion features in Docent. The POS distortion model slightly outperforms Moses with lexicalized reordering. This suggests that simple, soft, class-based distortion features can stand in for lexicalized reordering.

Key points

  • Docent's search operations: at each step, Docent randomly changes a phrase translation, swaps or moves phrases, or resegments phrases. It keeps the change only if the score improves, and it can be initialized from a Moses run.
  • English–French baseline: the system uses Moses plus Docent with three LMs: a 6-gram KenLM model, a 4-gram bilingual LM, and a 9-gram Brown-cluster LM built with SRILM. It reached 33.2 cased BLEU on newstest2014.
  • Anaphora model design: two binary neural networks are used, one choosing il/elle for "it" and one choosing ils/elles for "they".
    • Anaphora resolution runs separately as a preprocessing step.
    • Pronouns become placeholders in the training data and LM, so the anaphora model alone decides the pronoun choice.
  • Dependency projection model: a neural binary classifier separates good from bad alignments of source dependency arcs. Positive examples come from reference translations and negative examples from baseline n-best lists.
  • English–German distortion results: Docent used six soft POS phrase-distortion features, splitting phrases into verbs/particles versus all others. It scored 19.7 BLEU on both newstest2013 and newstest2014. Moses with lexicalized reordering scored 19.6 on both. Tuning used MERT on newstest2009.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Coreference resolution · Machine translation

Cited by

5 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 3 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/w14-3312

@inproceedings{hardmeier2014anaphora,
  title = {{Anaphora Models and Reordering for Phrase-Based SMT}},
  author = {Hardmeier, Christian and Stymne, Sara and Tiedemann, Jörg and Smith, Aaron and Nivre, Joakim},
  booktitle = {Proceedings of the Ninth Workshop on Statistical Machine Translation},
  year = {2014},
  doi = {10.3115/v1/w14-3312},
  url = {https://doi.org/10.3115/v1/w14-3312}
}

References

30 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Peter F. Brown, Peter V. deSouza, Robert L. Mercer, Vincent J. Della Pietra, and Jenifer C. Lai. 1992. Class-based n-gram models of natural language. Computational linguistics, 18(4):467–479. find
  2. Marine Carpuat, Yuval Marton, and Nizar Habash. 2010. Improving Arabic-to-English statistical machine translation by reordering post-verbal subjects for alignment. In Proceedings of the ACL 2010 Conference Short Papers, pages 178–183, Uppsala, Sweden. find
  3. Stanley F. Chen and Joshua Goodman. 1998. An empirical study of smoothing techniques for language modeling. Technical report, Computer Science Group, Harvard University, Cambridge, Massachusetts, USA. find our card
  4. Eunah Cho, Thanh-Le Ha, Mohammed Mediani, Jan Niehues, Teresa Herrmann, Isabel Slawik, and Alex Waibel. 2013. The Karlsruhe Institute of Technology translation systems for the WMT 2013. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 104–108, Sofia, Bulgaria. Associat… find
  5. Michael Collins, Philipp Koehn, and Ivona Kučerová. 2005. Clause restructuring for statistical machine translation. In Proceedings of the 43rd Annual Meeting of the ACL, pages 531–540, Ann Arbor, Michigan, USA. find our card
  6. Chris Dyer, Victor Chahuneau, and Noah A. Smith. 2013. A simple, fast, and effective reparameterization of IBM model 2. In Proceedings of the 2013 Conference of the NAACL: Human Language Technologies, pages 644–648, Atlanta, Georgia, USA. find
  7. Dmitriy Genzel. 2010. Automatically learning source-side reordering rules for large scale machine translation. In Proceedings of the 23rd International Conference on Computational Linguistics, pages 376– 384, Beijing, China. find
  8. Anita Gojun and Alexander Fraser. 2012. Determining the placement of German verbs in English–to– German SMT. In Proceedings of the 13th Conference of the EACL, pages 726–735, Avignon, France. find
  9. Christian Hardmeier, Joakim Nivre, and Jörg Tiedemann. 2012. Document-wide decoding for phrase-based statistical machine translation. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 1179– 1190, Jeju I… find
  10. Christian Hardmeier, Sara Stymne, Jörg Tiedemann, and Joakim Nivre. 2013a. Docent: A document-level decoder for phrase-based statistical machine translation. In Proceedings of the 51st Annual Meeting of the ACL, Demonstration session, pages 193– 198, Sofia, Bulgaria. find
  11. Christian Hardmeier, Jörg Tiedemann, and Joakim Nivre. 2013b. Latent anaphora resolution for cross-lingual pronoun prediction. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 380–391, Seattle, Washington, USA. Association for Computational Linguistic… find our card
  12. Kenneth Heafield. 2011. KenLM: faster and smaller language model queries. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 187–197, Edinburgh, Scotland. Association for Computational Linguistics. find our card
  13. Howard Johnson, Joel Martin, George Foster, and Roland Kuhn. 2007. Improving translation quality by discarding most of the phrasetable. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 967– 975, Prague,… find our card
  14. Philipp Koehn, Amittai Axelrod, Alexandra Birch Mayne, Chris Callison-Burch, Miles Osborne, and David Talbot. 2005. Edinburgh system description for the 2005 IWSLT speech translation evaluation. In Proceedings of the International Workshop on Spoken Language Translation, Pittsburgh, Pennsylvania, US… find our card
  15. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Proc… find
  16. Mohammed Mediani, Eunah Cho, Jan Niehues, Teresa Herrmann, and Alex Waibel. 2011. The KIT English–French translation systems for IWSLT 2011. In Proceedings of the International Workshop on Spoken Language Translation, pages 73–78, San Francisco, California, USA. find
  17. Hwidong Na, Jin-Ji Li, Jungi Kim, and Jong-Hyeok Lee. 2009. Improving fluency by reordering target constituents using MST parser in English-to-Japanese phrase-based SMT. In Proceedings of MT Summit XII, pages 276–283, Ottawa, Ontario, Canada. find
  18. Jan Niehues and Muntsin Kolss. 2009. A POS-based model for long-range reorderings in SMT. In Proceedings of the Fourth Workshop on Statistical Machine Translation, pages 206–214, Athens, Greece. find
  19. Jan Niehues, Teresa Herrmann, Stephan Vogel, and Alex Waibel. 2011. Wider context by using bilingual language models in machine translation. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 198–206, Edinburgh, Scotland. Association for Computational Linguistics. find
  20. Franz Josef Och, Nicola Ueffing, and Hermann Ney. 2001. An efficient A* search algorithm for Statistical Machine Translation. In Proceedings of the ACL 2001 Workshop on Data-Driven Machine Translation, pages 55–62, Toulouse, France. find
  21. Franz Josef Och. 2003. Minimum error rate training in statistical machine translation. In Proceedings of the 42nd Annual Meeting of the ACL, pages 160–167, Sapporo, Japan. find
  22. Maja Popović and Hermann Ney. 2006. POS-based reorderings for statistical machine translation. In Proceedings of the 5th International Conference on Language Resources and Evaluation (LREC’06), pages 1278–1283, Genoa, Italy. find
  23. Kay Rottmann and Stephan Vogel. 2007. Word reordering in statistical machine translation with a POS-based distortion model. In Proceedings of the 11th International Conference on Theoretical and Methodological Issues in Machine Translation, pages 171–180, Skövde, Sweden. find
  24. Andreas Stolcke. 2002. SRILM – an extensible language modeling toolkit. In Proceedings of the Seventh International Conference on Spoken Language Processing, pages 901–904, Denver, Colorado, USA. find
  25. Sara Stymne, Maria Holmqvist, and Lars Ahrenberg. 2010. Vs and OOVs: Two problems for translation between German and English. In Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR, pages 183–188, Uppsala, Sweden. find
  26. Sara Stymne, Christian Hardmeier, Jörg Tiedemann, and Joakim Nivre. 2013. Tunable distortion limits and corpus cleaning for SMT. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 225–231, Sofia, Bulgaria. find
  27. Katsuhito Sudoh, Xianchao Wu, Kevin Duh, Hajime Tsukada, and Masaaki Nagata. 2011. Post-ordering in statistical machine translation. In Proceedings of MT Summit XIII, pages 316–323, Xiamen. China. find
  28. Jörg Tiedemann. 2012. Parallel data, tools and interfaces in OPUS. In Proceedings of the 8th International Conference on Language Resources and Evaluation (LREC’12), pages 2214–2218, Istanbul, Turkey. find
  29. Ian H. Witten and Timothy C. Bell. 1991. The zero-frequency problem: Estimating the probabilities of novel events in adaptive text compression. IEEE Transactions on Information Theory, 37(4):1085– 1094. find
  30. Fei Xia and Michael McCord. 2004. Improving a statistical MT system with automatically learned rewrite patterns. In Proceedings of the 20th International Conference on Computational Linguistics, pages 508–514, Geneva, Switzerland. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/hardmeier-2014-anaphora-models-reordering-phrase-based/