Elamur Research⌕
☰
home / papers / 2004 / Improvements in Phrase-Based Statistical Machine Translation

Improvements in Phrase-Based Statistical Machine Translation

Summary

Zens and Ney present a phrase-based statistical machine translation system and test whether translating phrase by phrase, strictly left to right, can compete with stronger systems. Bilingual phrase pairs come from GIZA++ Viterbi alignments computed in both directions and merged by union. A pair is kept only if its words align inside the pair. Phrase probabilities are fractional relative frequencies. These are combined in a log-linear model with a bigram language model and three refinements: a word penalty, a phrase penalty, and a smoothed single-word lexicon. Feature weights are tuned against mWER on a development set. Decoding uses dynamic programming that is monotone at the phrase level, so its cost grows linearly with input length.

The system was evaluated on Verbmobil (German-English), Xerox (Spanish-English) and Canadian Hansards (French-English). On Xerox and Hansards it beat single-word-based systems, and on Hansards it also significantly beat the alignment template system. It translates over 1000 words per second on the smaller tasks. Verbmobil shows the limit of monotone search when word order differs a lot. The work shows that a simple, fast decoder can be enough for language pairs with similar word order.

Key points

  • Hansards results (5432 test sentences): the monotone phrase-based (PB) system scored WER 57.8, BLEU 27.8 and NIST 7.15. The alignment template (AT) system scored WER 61.1, BLEU 26.0 and NIST 6.71. The PB system was better on every metric, significant at the 99% level. Single-word baselines reached only BLEU 19.8–20.7.
  • Search complexity: worst case is O(J·M·V_e·E), where J is source length, M is maximum source phrase length, V_e is target vocabulary size and E is the number of translation candidates per phrase. On Hansards, translation time per sentence rose linearly with length, to about 1.5 s for 30-word sentences.
  • Speed on an AMD Athlon 2.2GHz:
    • Xerox: about 1448 words/s, under 7 s for a test set of over 10K words.
    • Verbmobil: about 1642 words/s, about 1.6 s for the whole test set.
    • Hansards: under 0.8 s per sentence on average.
  • Lexicon smoothing: the word lexicon uses absolute discounting with interpolation. It allows fractional counts and discount values above 1. The back-off distribution is either uniform or unigram.
  • Verbmobil (German-English): reordering mattered and monotone search was limited, but on mPER the system was competitive with the best system.

Interesting ideas

  • I1. Noisy-OR lexicon score. A source word's probability given a target phrase is taken as one minus the chance that no word in the phrase generates it. This gives a cheap way to back up sparse phrase statistics, since many long phrases occur only once.
  • I2. Two length-control features. A constant cost per output word controls translation length, and a constant cost per phrase controls how long the chosen phrases are. Both are trivial to compute and make useful knobs in any log-linear decoder.
  • I3. Fractional counting. When an item has several aligned partners, each partner receives a count of 1/N. Discounting is generalized so it still works on these non-integer counts. This can be applied to any count-based estimator built on noisy alignments.
  • I4. Filter the phrase table to the test input. The system loads only phrase pairs whose source side appears in the test data. This simple trick cuts memory use and speeds up translation.
  • I5. Measure monotonicity first. Checking how monotone a language pair is predicts whether a fast monotone decoder will be enough or reordering is required. This is a useful diagnostic before choosing system complexity, for example for a pair like Ukrainian-English.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Machine translation · Natural language processing

Topics: Machine translation

Cited by

98 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 93 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{zens2004improvements,
  title = {{Improvements in Phrase-Based Statistical Machine Translation}},
  author = {Zens, Richard and Ney, Hermann},
  booktitle = {Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004},
  year = {2004},
  url = {https://aclanthology.org/N04-1033/}
}

References

16 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. A. L. Berger, P. F. Brown, S. A. D. Pietra, V. J. D. Pietra, J. R. Gillett, A. S. Kehler, and R. L. Mercer. 1996. Language translation apparatus and method of using context-based translation models, United States patent, patent number 5510981, April. find
  2. P. F. Brown, J. Cocke, S. A. Della Pietra, V. J. Della Pietra, F. Jelinek, J. D. Lafferty, R. L. Mercer, and P. S. Roossin. 1990. A statistical approach to machine translation. Computational Linguistics, 16(2):79–85, June. find
  3. G. Doddington. 2002. Automatic evaluation of machine translation quality using n-gram co-occurrence statistics. In Proc. ARPA Workshop on Human Language Technology. find
  4. P. Koehn, F. J. Och, and D. Marcu. 2003. Statistical phrase-based translation. In Proc. of the Human Language Technology Conf. (HLT-NAACL), pages 127– 133, Edmonton, Canada, May/June. find
  5. D. Marcu and W. Wong. 2002. A phrase-based, joint probability model for statistical machine translation. In Proc. Conf. on Empirical Methods for Natural Language Processing, pages 133–139, Philadelphia, PA, July. find our card
  6. H. Ney, S. Martin, and F. Wessel. 1997. Statistical language modeling using leaving-one-out. In S. Young and G. Bloothooft, editors, Corpus-Based Methods in Language and Speech Processing, pages 174–207. Kluwer. find
  7. F. J. Och and H. Ney. 2002. Discriminative training and maximum entropy models for statistical machine translation. In Proc. of the 40th Annual Meeting of the Association for Computational Linguistics (ACL), pages 295–302, Philadelphia, PA, July. find our card
  8. F. J. Och, C. Tillmann, and H. Ney. 1999. Improved alignment models for statistical machine translation. In Proc. of the Joint SIGDAT Conf. on Empirical Methods in Natural Language Processing and Very Large Corpora, pages 20–28, University of Maryland, College Park, MD, June. find our card
  9. F. J. Och. 2003. Minimum error rate training in statistical machine translation. In Proc. of the 41th Annual Meeting of the Association for Computational Linguistics (ACL), pages 160–167, Sapporo, Japan, July. find
  10. K. A. Papineni, S. Roukos, T. Ward, and W. J. Zhu. 2001. Bleu: a method for automatic evaluation of machine translation. Technical Report RC22176 (W0109-022), IBM Research Division, Thomas J. Watson Research Center, September. find our card
  11. J. Pearl. 1988. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann Publishers, Inc., San Mateo, CA. Revised second printing. find
  12. W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery. 2002. Numerical Recipes in C++. Cambridge University Press, Cambridge, UK. find
  13. C. Tillmann and H. Ney. 2003. Word reordering and a dynamic programming beam search algorithm for statistical machine translation. Computational Linguistics, 29(1):97–133, March. find
  14. J. Tomas and F. Casacuberta. 2003. Combining phrase-based and template-based aligned models in statistical translation. In Proc. of the First Iberian Conf. on Pattern Recognition and Image Analysis, pages 1020– 1031, Mallorca, Spain, June. find
  15. W. Wahlster, editor. 2000. Verbmobil: Foundations of speech-to-speech translations. Springer Verlag, Berlin, Germany, July. find
  16. R. Zens, F. J. Och, and H. Ney. 2002. Phrase-based statistical machine translation. In 25th German Conference on Artificial Intelligence (KI2002), pages 18–32, Aachen, Germany, September. Springer Verlag. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/zens-2004-improvements-phrase-based-statistical-machine/