Elamur Research⌕
☰
home / papers / 2005 / Adaptation of the translation model for statistical machine translation based on information retrieval

Adaptation of the translation model for statistical machine translation based on information retrieval

Summary

The paper tackles domain mismatch in statistical machine translation. When large out-of-domain bilingual data is merged with a small in-domain corpus, the resulting translation model may pick word senses that are wrong for the test topic. The authors treat each source-side training sentence as a document and each test sentence as a query. They retrieve the most similar training sentences using TF-IDF cosine similarity and pool all retrieved pairs into one adapted corpus. They then retrain the translation model, and optionally the language model, on that corpus. To pick the selection size without a development set, they build language models on growing selections and choose the point where perplexity against the test sentence is lowest. They also re-rank retrieved batches by whether each batch lowers or raises perplexity. Experiments covered Spanish–English (medical) and Chinese–English (tourism). In both, adapted models beat strong baselines. On Chinese–English, perplexity-chosen sizes matched the best manually tuned results. This matters because it lets mixed or out-of-domain data help instead of hurt, with no separate in-domain tuning data needed.

Key points

  • Retrieval uses only the source side of the parallel data. No first-pass translation is needed, so selection does not depend on MT quality.
  • The system combines IBM1 lexicon and phrase transducers with a trigram Kneser-Ney LM built with the SRI toolkit. Results are scored with NIST and BLEU.
  • On Chinese–English, the in-domain baseline scored NIST 8.1129 / BLEU 0.4621. Perplexity-based size selection reached NIST 8.3812 / BLEU 0.4924. The best 3:1 in-/out-of-domain weighting gave NIST 8.3398 / BLEU 0.4931.
  • Perplexity re-ranking for Chinese–English used batches of 20 sentences. Because of runtime limits, only the top 800 retrieved sentences were re-ranked in the first pass and the top 600 in the second. Re-ranking itself brought little gain there.
  • Swapping TF-IDF for Okapi made no significant difference in translation quality. The two retrieval lists overlapped by more than 75% at the top 300 sentences and more than 90% at the top 1000.

Interesting ideas

  • I1. Test-set-driven data selection: Use each test sentence as a retrieval query and pool the hits into one shared training set, rather than building one model per sentence. This keeps the model robust and could be applied to tailoring any model to a known target batch, such as domain-specific Ukrainian translation jobs.
  • I2. Perplexity as a label-free stopping rule: Train a small LM on the top-k retrieved data and choose k at the perplexity minimum on the target text. This removes the need for a reference-annotated dev set, which helps in low-resource settings.
  • I3. Two-signal re-ranking: Bag-of-words retrieval ignores word order, while LM perplexity captures it. The authors mark each batch as "good" or "bad" by its perplexity effect and move bad batches to the end, keeping the original order within each group. This cheaply merges the two signals and could improve corpus filtering pipelines.
  • I4. Adapt the LM together with the TM: Training the language model on the target side of the same selected data keeps it from overriding topic-appropriate translations. The general lesson is to keep all model components consistent when adapting to a domain.
  • I5. Weighting in-domain versus retrieved data: Duplicating or up-weighting the small in-domain portion (for example 3:1) noticeably shifted the scores. When mixing scarce clean data with large retrieved data, this ratio is worth tuning explicitly.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Topics: Machine translation

Cited by

75 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 72 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{hildebrand2005adaptation,
  title = {{Adaptation of the translation model for statistical machine translation based on information retrieval}},
  author = {Hildebrand, Almut Silja and Eck, Matthias and Vogel, Stephan and Waibel, Alex},
  booktitle = {Proceedings of the 10th EAMT Conference: Practical applications of machine translation},
  year = {2005},
  doi = {10.5445/ir/1000182997},
  url = {https://doi.org/10.5445/ir/1000182997}
}

References

15 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. BROWN, Peter E., DELLA PIETRA, Stephen A., DELLA PIETRA, Vincent J. and MERCER, Robert L. (1993). ‘The mathematics of statistical machine translation: Parameter estimation’, Computational Linguistics, 19(2), pp. 263-311 find
  2. DODDINGTON, George (2001). ‘Automatic Evaluation of Machine Translation Quality using n-Gram Co-occurrence Statistics’. NIST Washington, DC, USA. find
  3. ECK, Matthias, VOGEL, Stephan and WAIBEL, Alex (2004). ‘Language Model Adaptation for Statistical Machine Translation based on Information Retrieval’, Proceedings of LREC 2004, Lisbon, Portugal, May 2004. find
  4. KIM, Woosung and KHUDANPUR, Sanjeev (2003). ‘Language Model Adaptation Using Cross-Lingual Information’, Proceedings of Eurospeech 2003, Geneva, Switzerland, Sept. 2003. find
  5. ‘The LEMUR Toolkit for Language Modeling and Information Retrieval’ http://www.cs.cmu.edu/~lemur/ ↗
  6. MAHAJAN, Milind BEEFERMAN Doug and HUANG, X.D. (1999). ‘Improved Topic-Dependent Language Modeling Using Information Retrieval Techniques’, IEEE International Conference on Acoustics, Speech and Signal Processing 1999, Phoenix, AZ. find
  7. PAPINENI, Kishore, ROUKOS, Salim, WARD, Todd and ZHU, Wei-Jing (2002). ‘BLEU: a Method for Automatic Evaluation of Machine Translation’, Proceedings of the ACL 2002, Philadelphia, USA. find our card
  8. TAKEZAWA, Toshiyuki, SUMITA, Eiichiro, SU-GAYA, Fumiaki, YAMAMOTO, Hirofumi, YA-MAMOTO, Seiichi (2002) ‘Toward a Broad-coverage Bilingual Corpus for Speech Translation of Travel Conversation in the Real World’, LREC 2002 (Third International Conference on Language Resources and Evaluation), Vol.1, p… find
  9. VOGEL, Stephan, NEY, Hermann and TILLMANN, Christoph (1996). ‘HMM-based Word Alignment in Statistical Translation’, Proceedings of COLING 1996: Proceedings of Coling 1996. find
  10. Copenhagen, August 1996. find
  11. VOGEL, Stephan, ZHANG, Ying, HUANG, Fei, TRIBBLE, Alicia, VENUGOPAL, Ashish, ZHAO, Bing, WAIBEL, Alex (2003). ‘The CMU Statistical Translation System’, Proceedings of MT-Summit IX 2003. find
  12. New Orleans, LA. Sept. 2003. find
  13. WU, Hua and WANG, Haifeng (2004). ‘Improving Domain-Specific Word Alignment for Computer Assisted Translation’, Proceedings of ACL 2004, Barcelona, Spain, July 2004. find
  14. ZHANG, Ying, VOGEL, Stephan and WAIBEL, Alex (2003). ‘Integrated Phrase Segmentation and Alignment Algorithm for Statistical Machine Translation’, Proceedings of International Conference on Natural Language Processing and Knowledge Engineering 2003, Beijing, China, Oct. 2003. find
  15. ZHAO, Bing, ECK, Matthias and VOGEL, Stephan 2004. ‘Language Model Adaptation for Statistical Machine Translation via Structured Query Models’, Proceedings of Coling 2004, Geneva, Switzerland, Aug. 2004. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/hildebrand-2005-adaptation-translation-model-statistical-machine/