Elamur Research⌕
☰
home / papers / 2006 / Phrasetable Smoothing for Statistical Machine Translation

Phrasetable Smoothing for Statistical Machine Translation

Paper · 2006 · George Foster, Roland Kuhn and Howard Johnson

Summary

Phrase-based SMT systems usually estimate phrase translation probabilities with plain relative frequencies (RF). This overfits: a pair whose phrases each appear only once gets probability 1, which beats better-supported pairs during decoding. The authors present what they describe as the first systematic comparison of phrasetable smoothing methods. They split these into two families. "Black-box" methods treat phrases as atomic units and adapt LM-style techniques such as Good-Turing, fixed-discount and Kneser-Ney. "Glass-box" methods decompose phrases into words, as in Zens-Ney noisy-or and Koehn-Och-Marcu IBM1 smoothing. They also test combinations of the two families.

They evaluated on six WMT06 Europarl directions with small corpora and on large-scale NIST MT05 Chinese-English. Every smoothing method beat RF. In the European setting, Kneser-Ney combined with Zens-Ney gave gains of roughly 1–2 BLEU. For Chinese-English, Good-Turing and glass-box combinations did best. The study shows that a cheap, often ignored step yields consistent translation gains.

Key points

  • Data:
    • WMT06 Europarl: about 700,000 sentence pairs each for French, Spanish and German with English, plus a 2000-sentence dev set.
    • NIST MT05 Chinese-English: 4.98M UN sentences and 3.16M non-UN sentences, each used for a separate phrasetable. A Gigaword subset of 11.7M sentences was added for the LM.
  • Significance testing: 1000-fold pairwise bootstrap resampling at 95% confidence.
  • European results:
    • Any smoothing improved over RF by at least 0.45 BLEU.
    • Kneser-Ney beat Good-Turing by at least 0.30 BLEU.
    • KN3+ZN-IBM1 scored best, for example fr→en 25.35 → 27.25 and en→fr 27.20 → 29.00.
  • Chinese-English results:
    • RF scored 29.85 and GT scored 30.66, statistically tied with KN3 at 30.54.
    • The best scores came from RF+ZN-IBM1 (30.95) and GT+KOM-IBM1 (30.81).
    • Glass-box methods used alone did poorly: ZN-IBM1 scored 29.55 and KOM-IBM1 scored 28.09.
  • Caveat: glass-box weights were tuned for BLEU with Och's algorithm, while black-box linear weights were set heuristically. This gave glass-box methods an advantage in the comparison.

Interesting ideas

  • I1. Treat a phrase pair like a bigram. Doing so lets well-understood LM smoothing (GT, KN) be reused directly on a new kind of count table. This is handy whenever you have sparse co-occurrence counts but no natural lower-order model.
  • I2. Prefer interpolation over backoff when unseen events never matter. The decoder only queries pairs that were seen, so the main benefit of smoothing is reshaping estimates for observed items. Interpolation does this; backoff does not. The same logic applies to any lookup-only probability table.
  • I3. Loglinear mixing with a uniform distribution is not smoothing. A simple Bernoulli example shows it keeps 0 and 1 intact and pushes middle values toward the extremes. This is a useful sanity check before calling a loglinear combination "smoothing."
  • I4. Combine both families. Black-box and glass-box smoothing appear to be complementary, so stacking a count-based discount with a word-level lexical model can beat either one alone.
  • I5. Use learning curves for size-dependent effects. When a method helps in one data regime but not another, training on corpus subsets can reveal whether size is the cause. Here the curves also exposed high sensitivity to which corpus sample was used.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏛️National Research Council CanadaOttawa · Canada40 works · 20,943 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,085 worksArtificial Intelligence · 8,888 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Machine translation · 958 works

Cite

Cite as George Foster, Roland Kuhn and Howard Johnson. 2006. Phrasetable Smoothing for Statistical Machine Translation. Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{foster2006phrasetable,
  title = {{Phrasetable Smoothing for Statistical Machine Translation}},
  author = {George Foster, Roland Kuhn and Howard Johnson},
  booktitle = {Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing},
  year = {2006},
  doi = {10.3115/1610075.1610084},
  url = {https://doi.org/10.3115/1610075.1610084}
}

References

22 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Peter F. Brown, Stephen A. Della Pietra, Vincent Della J. Pietra, and Robert L. Mercer. 1993. The mathematics of Machine Translation: Parameter estimation. Computational Linguistics, 19(2):263– 312, June. find
  2. M. Cettollo, M. Federico, N. Bertoldi, R. Cattoni, and B. Chen. 2005. A look inside the ITC-irst SMT system. In Proceedings of MT Summit X, Phuket, Thailand, September. International Association for Machine Translation. find
  3. Stanley F. Chen and Joshua T. Goodman. 1998. An empirical study of smoothing techniques for language modeling. Technical Report TR-10-98, Computer Science Group, Harvard University.
  4. K. Church and W. Gale. 1991. A comparison of the enhanced Good-Turing and deleted estimation methods for estimating probabilities of English bigrams. Computer speech and language, 5(1):19–54. find
  5. M. Collins and J. Brooks. 1995. Prepositional phrase attachment through a backed-off model. In Proceedings of the 3rd ACL Workshop on Very Large Corpora (WVLC), Cambridge, Massachusetts.
  6. Ismael Garcı́a-Varea, Francisco Casacuberta, and Hermann Ney. 1998. An iterative, DP-based search algorithm for statistical machine translation. In Proceedings of the 5th International Conference on Spoken Language Processing (ICSLP) 1998, volume 4, pages 1135–1138, Sydney, Australia, December. find
  7. Joshua Goodman. 2001. A bit of progress in language modeling. Computer Speech and Language. find
  8. Trevor Hastie, Robert Tibshirani, and Jerome Friedman. 2001. The Elements of Statistical Learning. Springer. find
  9. Reinhard Kneser and Hermann Ney. 1995. Improved backing-off for m-gram language modeling. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 1995, pages 181–184, Detroit, Michigan. IEEE. find
  10. Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Eduard Hovy, editor, Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, pages 127–133, Edmonton, Alberta, Canada, May.… find
  11. P. Koehn, A. Axelrod, A. B. Mayne, C. Callison-Burch, M. Osborne, D. Talbot, and M. White. 2005. Edinburgh system description for the 2005 NIST MT evaluation. In Proceedings of Machine Translation Evaluation Workshop. find
  12. Philippe Langlais, Guihong Cao, and Fabrizio Gotti. 2005. RALI: SMT shared task system description. find
  13. In Proceedings of the 2nd ACL workshop on Building and Using Parallel Texts, pages 137–140, University of Michigan, Ann Arbor, June. find
  14. Daniel Marcu and William Wong. 2002. A phrase-based, joint probability model for statistical machine translation. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing (EMNLP), Philadelphia, PA.
  15. Robert C. Moore. 2004. Improving IBM word-alignment model 1. In Proceedings of the 42th Annual Meeting of the Association for Computational Linguistics (ACL), Barcelona, July. find
  16. Hermann Ney, Ute Essen, and Reinhard Kneser. 1994. On structuring probabilistic dependencies in stochastic language modelling. Computer Speech and Language, 10:1–38. find
  17. Arthur Nádas. 1985. On Turing’s formula for word probabilities. IEEE Transactions on Acoustics, Speech and Signal Processing (ASSP), ASSP-33(6):1415–1417, December. find
  18. Franz Josef Och. 2003. Minimum error rate training for statistical machine translation. In Proceedings of the 41th Annual Meeting of the Association for Computational Linguistics (ACL), Sapporo, July. find
  19. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2001. BLEU: A method for automatic evaluation of Machine Translation. Technical Report RC22176, IBM, September.
  20. Andreas Stolcke. 2002. SRILM - an extensible language modeling toolkit. In Proceedings of the 7th International Conference on Spoken Language Processing (ICSLP) 2002, Denver, Colorado, September. find
  21. WMT. 2006. The NAACL Workshop on Statistical Machine Translation (www.statmt.org/wmt06), New York, June. find
  22. Richard Zens and Hermann Ney. 2004. Improvements in phrase-based statistical machine translation. In Proceedings of Human Language Technology Conference / North American Chapter of the ACL, Boston, May.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/foster-2006-phrasetable-smoothing-statistical-machine-translation/