Elamur Research⌕
☰
home / papers / 2007 / Pivot Language Approach for Phrase-Based Statistical Machine Translation

Pivot Language Approach for Phrase-Based Statistical Machine Translation

Summary

The paper tackles phrase-based SMT for language pairs with little or no direct parallel data. The authors route through a third, well-resourced pivot language. They train separate source–pivot and pivot–target phrase models and combine them into a source–target "pivot model". Phrase probabilities are obtained by summing over shared pivot phrases. Lexical weights are recomputed from word alignments projected through the pivot. When a small direct corpus exists, the pivot model is linearly interpolated with the standard model, and several pivot languages can be mixed.

Experiments use Europarl data from the NAACL/HLT 2006 shared task, with Koehn's training scripts and the Pharaoh decoder. They cover French→Spanish, French→German and French→English, plus Chinese→Japanese with English as pivot. For French–Spanish, an English+German pivot model gains 0.06 BLEU (22.13% relative) over a model trained on 5,000 direct sentence pairs. This roughly matches a direct model trained on 30,000 pairs.

The approach needs no language-specific tools, so it is attractive wherever English-centric corpora are plentiful but direct ones are not.

Key points

  • The pivot phrase probability marginalizes over pivot phrases: φ(f|e) = Σ_p φ(f|p)·φ(p|e). This assumes f is independent of e given p.
  • Two variants estimate the lexical weight:
    • "Phrase method": counts come from alignments induced through the pivot, weighted by phrase probabilities.
    • "Word method": follows Wang et al. (2006).
  • With 5,000 French–Spanish pairs, interpolation lifts BLEU from 0.2747 to 0.3229 (17.55% relative). The gain shrinks to about 0.01 BLEU at 50,000 pairs and disappears at around 100,000.
  • Pivot model BLEU for French–Spanish depends on the pivot language:
    • English alone: 0.3212
    • German alone: 0.3077
    • Both combined (weights 0.6/0.4, tuned on the dev set): 0.3355
  • Pivot corpus size matters. Pivot models trained on 200k, 400k and the full corpora score 0.2376, 0.2954 and 0.3212 respectively.
  • Chinese–Japanese is a weaker case. It uses 21,977 direct pairs, 329,350 Chinese–English pairs and 160,535 English–Japanese pairs. Here the pivot model only beats a standard model trained on 2,500 pairs. The authors attribute this to smaller pivot corpora, open-domain data, and less related languages.

Interesting ideas

  • I1. Phrase-table triangulation: Joining two phrase tables on the pivot side yields a usable model for an unseen language pair. This could help pairs such as Ukrainian–X, where only Ukrainian–English and English–X data are plentiful.
  • I2. Alignment projection through the pivot: Word links are composed through the pivot so that features needing alignments, like lexical weights, can be rebuilt. The same trick applies to any feature that depends on token-level correspondences.
  • I3. Interpolation instead of replacement: Linearly mixing the scarce direct model with pivot-derived models lets each contribute where it is strong. The paper's own curves show the benefit fading as direct data grows, which gives a practical guide for when pivoting is worth the effort.
  • I4. Combining multiple pivots: Using several pivots with dev-tuned weights helps because different pivot languages capture different phenomena. Ensembling over pivots is a cheap way to reduce pivot-specific errors.
  • I5. Evaluation by equivalent direct data: The paper reports how much direct parallel data the pivot model is "worth" (here, about 30k pairs). This is an intuitive way to communicate the value of any data-augmentation method.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Machine translation

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{wu2007pivot,
  title = {{Pivot Language Approach for Phrase-Based Statistical Machine Translation}},
  author = {Wu, Hua and Wang, Haifeng},
  booktitle = {Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics},
  year = {2007},
  doi = {10.1007/s10590-008-9041-6},
  url = {https://doi.org/10.1007/s10590-008-9041-6}
}

References

29 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hiyan Alshawi, Srinivas Bangalore, and Shona Douglas. 2000. Learning Dependency Translation Models as Collections of Finite-State Head Transducers. Computational Linguistics, 26(1):45-60. find
  2. Niraj Aswani and Robert Gaizauskas. 2005. Aligning Words in English-Hindi Parallel Corpora. In Proc. of the ACL 2005 Workshop on Building and Using Parallel Texts: Data-driven Machine Translation and Beyond, pages 115-118. find
  3. Peter F. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993. The Mathematics of Statistical Machine Translation: Parameter Estimation. Computational Linguistics, 19(2): 263-311. find
  4. Chris Callison-Burch, Philipp Koehn, and Miles Osborne. 2006. Improved Statistical Machine Translation Using Paraphrases. In Proc. of NAACL-2006, pages 17-24. find
  5. Lars Borin. 2000. You'll Take the High Road and I'll Take the Low Road: Using a Third Language to Improve Bilingual Word Alignment. In Proc. of COL-ING-2000, pages 97-103. find
  6. David Chiang. 2005. A Hierarchical Phrase-Based Model for Statistical Machine Translation. In Proc. of ACL-2005, pages 263-270. find our card
  7. Mona Diab and Philip Resnik. 2002. An Unsupervised Method for Word Sense Tagging using Parallel Corpora. In Proc. of ACL-2002, pages 255-262. find
  8. Tim Gollins and Mark Sanderson. 2001. Improving Cross Language Information Retrieval with Triangulated Translation. In Proc. of ACM SIGIR-2001, pages 90-95. find
  9. Kazuaki Kishida and Noriko Kando. 2003. Two-Stage Refinement of Query Translation in a Pivot Language Approach to Cross-Lingual Information Retrieval: An Experiment at CLEF 2003. In Proc. of CLEF-2003. pages 253-262. find
  10. Philipp Koehn. 2004. Pharaoh: A Beam Search Decoder for Phrase-Based Statistical Machine Translation Models. In Proc. of AMTA-2004, pages 115-124. find
  11. Philipp Koehn. 2005. Europarl: A Parallel Corpus for Statistical Machine Translation. In Proc. of MT Summit X, pages 79-86. find
  12. Philipp Koehn and Christof Monz. 2006. Manual and Automatic Evaluation of Machine Translation between European Languages. In Proc. of the 2006 HLT-NAACL Workshop on Statistical Machine Translation, pages 102-121. find our card
  13. Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical Phrase-Based Translation. In Proc. of HLT-NAAC- 2003, pages 127-133. find
  14. Adam Lopez and Philip Resnik. 2005. Improved HMM Alignment Models for Languages with Scarce Resources. In Proc. of the ACL-2005 Work-shop on Building and Using Parallel Texts: Data-driven Machine Translation and Beyond, pages 83-86. find
  15. Joel Martin, Rada Mihalcea, and Ted Pedersen. 2005. Word Alignment for Languages with Scarce Resources. In Proc. of the ACL-2005 Workshop on Building and Using Parallel Texts: Data-driven Machine Translation and Beyond, pages 65-74. find
  16. Dan Melamed. 2004. Statistical Machine Translation by Parsing. In Proc. of ACL-2004, pages 653-660. find
  17. Bart Mellebeek, Karolina Owczarzak, Declan Groves, Josef Van Genabith, and Andy Way. 2006. A Syntactic Skeleton for Statistical Machine Translation. In Proc. of EAMT-2006, pages 195-202. find
  18. Sonja Niessen and Hermann Ney. 2004. Statistical Machine Translation with Scarce Resources Using Morpho-Syntactic Information. Computational linguistics, 30(2): 181-204. find our card
  19. Franz Josef Och. 2003. Minimum Error Rate Training in Statistical Machine Translation. In Proc. of ACL-2003, pages 160-167. find
  20. Franz Josef Och and Hermann Ney. 2004. The Alignment Template Approach to Statistical Machine Translation. Computational Linguistics, 30(4):417-449. find
  21. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. B LEU : a Method for Automatic Evaluation of Machine Translation. In Proc. of ACL-2002, pages 311-318. find
  22. Chris Quirk, Arul Menezes, and Colin Cherry. 2005. Dependency Treelet Translation: Syntactically Informed Phrasal SMT. In Proc. of ACL-2005, pages 271-279. find our card
  23. Tetsuya Sakai, Makoto Koyama, Akira Kumano, and Toshihiko Manabe. 2004. Toshiba BRIDJE at NTCIR-4 CLIR: Monolingual/Bilingual IR and Flexible Feedback. In Proc. of NTCIR 4. find
  24. Charles Schafer and David Yarowsky. 2002. Inducing Translation Lexicons via Diverse Similarity Measures and Bridge Languages. In Proc. of CoNLL-2002, pages 1-7. find
  25. Haifeng Wang, Hua Wu, and Zhanyi Liu. 2006. Word Alignment for Languages with Scarce Resources Using Bilingual Corpora of Other Language Pairs. In Proc. of COLING/ACL-2006 Main Conference Poster Sessions, pages 874-881. find
  26. Dan Tufis, Radu Ion, Alexandru Ceausu, and Dan Stefanescu. 2005. Combined Word Alignments. In Proc. of the ACL-2005 Workshop on Building and Using Parallel Texts: Data-driven Machine Translation and Beyond, pages 107-110. find
  27. Vincent Vandeghinste, Ineka Schuurman, Michael Carl, Stella Markantonatou, and Toni Badia. 2006. METIS-II: Machine Translation for Low-Resource Languages. In Proc. of LREC-2006, pages 1284-1289. find
  28. Dekai Wu. 1997. Stochastic Inversion Transduction Grammars and Bilingual Parsing of Parallel Corpora. Computational Linguistics, 23(3):377-403. find our card
  29. Kenji Yamada and Kevin Knight. 2001. A Syntax Based Statistical Translation Model. In Proc. of ACL-2001, pages 523-530. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wu-2007-pivot-language-approach-phrase-based/