Elamur Research⌕
☰
home / papers / 2007 / Improving Statistical Machine Translation Performance by Training Data Selection and Optimization

Improving Statistical Machine Translation Performance by Training Data Selection and Optimization

Paper · 2007 · Yajuan Lü, Jin Huang and Qun Liu

Summary

The paper tackles a common situation in Chinese–English statistical machine translation. Parallel data from LDC comes from mixed domains such as Hong Kong laws, Hansards and news. Simply pooling these corpora does not reliably improve translation, and adding ever more data brings shrinking gains at growing computational cost. The authors therefore try to get more out of existing corpora rather than collect new ones, and propose two methods.

  • Offline method (test domain known): TF-IDF sentence retrieval with the Lemur toolkit finds training pairs resembling the test text. These pairs either form a compact adapted corpus or are given extra weight within the full corpus.
  • Online method (test domain unknown): separate submodels are trained per sub-corpus alongside a general model. For each incoming sentence, retrieval results decide how the submodels are combined in a log-linear interpolation.

Both methods improve BLEU without any extra resources. The work shows that relevance of training data matters alongside sheer quantity.

Key points

  • Retrieval indexes only the source-language side of the training data. Each training sentence is a document and each test sentence is a query. Because no first-pass translation is needed, the method does not depend on the quality of a baseline system.
  • The top N retrieved pairs, with N ranging from one to several thousand, form the adapted set. Duplicates across queries can be kept or removed, and the paper compares both options.
  • Reweighting is done by adjusting the occurrence count in GIZA++'s three-line training file format. Model size stays the same as the original, and no extra training time or memory is required.
  • The online setup uses FBIS, HK_Hansards and HK_News as submodels and the full baseline model as the general model. It compares four weighting schemes, two of which fall back to the general model when no submodel accounts for more than half of the retrieved sentences.
  • On a mixed single-reference test set (including an MT05 part), all four schemes score about 0.1034–0.1038 BLEU on the whole set. They beat the baseline overall and on every sub-part, though the schemes barely differ from each other.

Interesting ideas

  • I1. Upweight instead of filter: adding the retrieved in-domain subset on top of the full corpus boosts relevant pairs while keeping all the data. This avoids the sparsity that comes with training on a small selected subset, and the same idea fits any low-resource domain adaptation.
  • I2. Weighting through a count field: encoding sentence weights as repetition counts in the aligner's input file gives domain weighting at essentially no engineering cost. Any toolkit that accepts per-example counts or weights could be adapted the same way.
  • I3. Per-sentence model mixing: retrieving neighbours for each input and reading off which sub-corpus they come from gives a cheap, unsupervised domain classifier. It can drive model interpolation at inference time, which is useful when input domains are mixed or unknown.
  • I4. Fallback threshold: using the general model unless one submodel clearly dominates (over 0.5 of the retrieved sentences) is a simple safeguard against confident misrouting. It is a sensible default for any mixture-of-experts style routing.
  • I5. Small adapted models for constrained settings: a compact retrieved corpus can roughly match a large-corpus baseline while being much smaller. This makes it attractive for memory-limited deployments or quick domain prototypes.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Machine translation · 958 works

Cite

Cite as Yajuan Lü, Jin Huang and Qun Liu. 2007. Improving Statistical Machine Translation Performance by Training Data Selection and Optimization. Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{lu2007improving,
  title = {{Improving Statistical Machine Translation Performance by Training Data Selection and Optimization}},
  author = {Yajuan Lü, Jin Huang and Qun Liu},
  booktitle = {Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL)},
  year = {2007},
  url = {https://aclanthology.org/D07-1036/}
}

References

17 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jisong Chen, Rowena Chau, Chung-Hsing Yeh 2004. Discovering Parallel Text from the World Wide Web. find
  2. Stanley F. Chen and Joshua Goodman. 1998. An Empirical Study of Smoothing Techniques for Language Modeling. Technical Report TR-10-98, Harvard University Center for Research in Computing Technology.
  3. Matthias Eck, Stephan Vogel, and Alex Waibel 2004. Language M odel A daptation for S tatistical M achine T ranslation B ased on I nformation R etrieval. Proceedings of Fourth International Conference on Language Resources and Evaluation:327-330 find
  4. Matthias Eck, Stephan Vogel, Alex Waibel 2005. Low cost portability for statistical machine translation based on n-gram coverage. MT Summit X: 227-234. find
  5. Zhongjun He, Yang Liu, Deyi Xiong, Hongxu Hou, and Qun Liu 2006. ICT System Description for the 2006 TC-STAR Run\#2 SLT Evaluation. Proceedings of TC-STAR Workshop on Speech-to-Speech Translation: 63-68 find
  6. Philipp Koehn, Franz J. Och, and Daniel Marcu. 2003. Statistical phrase-based translation. Proceedings of HLT-NAACL 2003: 127–133. find
  7. Arne Mauser, Richard Zens, Evgeny Matusov, Sasa Hasan, Hermann Ney 2006. The RWTH Statistical Machine Translation System for the IWSLT 2006 Evaluation. Proceedings of International Workshop on Spoken Language Translation.:103-110 find
  8. Dragos Stefan Munteanu and Daniel Marcu 2005. Improving Machine Translation Performance by Exploiting Comparable Corpora. Computational Linguistics, 31 (4): 477-504 find
  9. Dragos Stefan Munteanu and Daniel Marcu 2006. Extracting Parallel Sub-Sentential Fragments from Comparable Corpora. ACL-2006: 81-88 find
  10. Jian-Yun Nie, Michel Simard, Pierre Isabelle, Richard Durand 1999. Cross-Language Information Retrieval based on Parallel Texts and Automatic Mining of Parallel Texts in the Web. SIGIR-1999: 74-81 find
  11. Franz Josef Och 2003. Minimum Error Rate Training in Statistical Machine Translation. ACL-2003:160-167. find
  12. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a Method for Automatic Evaluation of Machine Translation. ACL-2002: 311– 318
  13. Philip Resnik and Noah A. Smith 2003. The Web as a Parallel Corpus. Computational Linguistics 29(3): 349-380
  14. Almut Silja Hildebrand, Matthias Eck, Stephan Vogel, and Alex Waibel 2005. Adaptation of the Translation Model for Statistical Machine Translation based on Information Retrieval. Proceedings of EAMT 2005: 133-142.
  15. Richard Zens, Franz Josef Och, Hermann Ney 2002. Phrase-Based Statistical Machine Translation. Annual German Conference on AI, KI 2002, Vol. LNAI 2479: 18-32 find
  16. Ying Zhang, Almut Silja Hildebrand, Stephan Vogel 2006. Distributed Language Modeling for N-best List Re-ranking. EMNLP-2006:216-223 find
  17. Bing Zhao, Matthias Eck, Stephan Vogel 2004. Language Model Adaptation for Statistical Machine Translation with structured query models. COLING-2004 find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lu-2007-improving-statistical-machine-translation-performance/