Elamur Research⌕
☰
home / papers / 2007 / Word Sense Disambiguation Improves Statistical Machine Translation

Word Sense Disambiguation Improves Statistical Machine Translation

Summary

Earlier studies disagreed on whether word sense disambiguation (WSD) actually helps statistical machine translation. Some reported losses, and others showed gains only on simplified tasks or without significance testing. The authors test the question on a real Chinese-to-English translation task by plugging a strong WSD classifier into Hiero, a hierarchical phrase-based system built on synchronous CFG.

Instead of using a fixed sense inventory, the WSD "senses" are English phrases aligned to Chinese words or two-word phrases in the parallel corpus. Training examples are harvested at the moments when Hiero extracts its rules. The classifier is an SVM that uses local collocations, POS tags and surrounding words.

Its predictions enter Hiero as two extra log-linear features attached to existing grammar rules, so no competing rules are created. These features are tuned and decoded together with all the other features.

The combined Hiero+WSD system translates measurably better, and the gain is statistically significant. This is the first clear evidence that WSD can benefit a state-of-the-art MT system.

Key points

  • The WSD classifier is an SVM (implementation of Chang and Lin, 2001) that outputs a probability for each class. It uses three groups of features:
    • local collocations (w−1 w+1, w−1, w+1);
    • POS tags (P−1, P0, P+1);
    • surrounding unigrams, kept only if they appear at least 3 times with some sense.
  • On the SENSEVAL-3 Chinese lexical-sample test data, the classifier's accuracy compares favourably with the best system that took part in that task.
  • Hiero is extended with two new features:
    • P_wsd(t|s), the contextual probability the classifier gives to translating source substring s as target substring t;
    • P_ty_wsd = exp(−|t|), which rewards rules that contain WSD-suggested translations.
  • Training data for the WSD model is built as follows:
    • source phrases are limited to 1–2 Chinese words;
    • each example's context is its sentence plus one neighbouring sentence on each side;
    • translations seen only once are dropped;
    • a phrase is classified only if it has at least 10 examples.
  • The Collins sign-test judged 374 test sentences better and 318 worse with Hiero+WSD. About 73% of WSD-proposed translations appear in the reference translations, and of the 7087 single-word proposals, 77.31% match.

Interesting ideas

  • I1. Translations as senses. Phrases aligned in a parallel corpus serve as the sense labels, so no hand-built sense inventory is needed. This could transfer to Ukrainian–English work where resources like WordNet are thin.
  • I2. Score existing rules instead of adding new ones. The WSD output only adds feature scores to rules the system already has. External knowledge (a classifier, a gazetteer, a terminology list) can then be weighed by standard tuning instead of a manually chosen weight.
  • I3. Pair a sparse feature with a companion indicator. A probability feature that fires on only some rules would penalise those rules. A second, count-like feature cancels that bias, which is a reusable trick for any partial-coverage signal in log-linear models.
  • I4. Source-side context comes almost free. Because the input sentence is fixed, features conditioned on arbitrary source context leave decoding complexity unchanged. Rich source-side classifiers can therefore be added cheaply.
  • I5. Look past the corpus-level score. Using a per-sentence sign-test and inspecting the worse outputs showed that some "losses" were adequate translations that BLEU penalised for wording. This is a useful reminder to complement precision-based metrics with manual error analysis.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Word sense disambiguation · Part-of-speech tagging · Machine translation

Cited by

137 papers of our library cite it (of the 140884 whose bibliographies we read); 4 of them have a page here, 133 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{chan2007word,
  title = {{Word Sense Disambiguation Improves Statistical Machine Translation}},
  author = {Chan, Yee Seng and Ng, Hwee Tou and Chiang, David},
  booktitle = {Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics},
  year = {2007},
  url = {https://aclanthology.org/P07-1005/}
}

References

23 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. C. Cabezas and P. Resnik. 2005. Using WSD techniques for lexical selection in statistical machine translation. Technical report, University of Maryland. find
  2. M. Carpuat and D. Wu. 2005. Word sense disambiguation vs. statistical machine translation. In Proc. of ACL05, pages 387–394. find
  3. M. Carpuat, W. Su, and D. Wu. 2004. Augmenting ensemble classification for word sense disambiguation with a kernel PCA model. In Proc. of SENSEVAL-3, pages 88–92. find
  4. C. C. Chang and C. J. Lin, 2001. LIBSVM: a library for support vector machines. Software available at http://www. csie.ntu.edu.tw/˜cjlin/libsvm. find
  5. D. Chiang. 2005. A hierarchical phrase-based model for statistical machine translation. In Proc. of ACL05, pages 263– 270. find our card
  6. D. Chiang. 2007. Hierarchical phrase-based translation. To appear in Computational Linguistics, 33(2). find
  7. M. Collins, P. Koehn, and I. Kucerova. 2005. Clause restructuring for statistical machine translation. In Proc. of ACL05, pages 531–540. find our card
  8. U. Germann. 2003. Greedy decoding for statistical machine translation in almost linear time. In Proc. of HLT-NAACL03, pages 72–79. find
  9. P. Koehn, F. J. Och, and D. Marcu. 2003. Statistical phrase-based translation. In Proc. of HLT-NAACL03, pages 48–54. find
  10. P. Koehn. 2003. Noun Phrase Translation. Ph.D. thesis, University of Southern California. find
  11. P. Koehn. 2004a. Pharaoh: A beam search decoder for phrase-based statistical machine translation models. In Proc. of AMTA04, pages 115–124. find
  12. P. Koehn. 2004b. Statistical significance tests for machine translation evaluation. In Proc. of EMNLP04, pages 388– 395. find
  13. Y. K. Lee and H. T. Ng. 2002. An empirical evaluation of knowledge sources and learning algorithms for word sense disambiguation. In Proc. of EMNLP02, pages 41–48. find
  14. P. M. II Lewis and R. E. Stearns. 1968. Syntax-directed transduction. Journal of the ACM, 15(3):465–488. find
  15. D. Marcu and W. Wong. 2002. A phrase-based, joint probability model for statistical machine translation. In Proc. of EMNLP02, pages 133–139. find our card
  16. F. J. Och and H. Ney. 2000. Improved statistical alignment models. In Proc. of ACL00, pages 440–447. find our card
  17. F. J. Och and H. Ney. 2002. Discriminative training and maximum entropy models for statistical machine translation. In Proc. of ACL02, pages 295–302. find our card
  18. F. J. Och and H. Ney. 2004. The alignment template approach to statistical machine translation. Computational Linguistics, 30(4):417–449. find
  19. F. J. Och. 2003. Minimum error rate training in statistical machine translation. In Proc. of ACL03, pages 160–167. find
  20. K. Papineni, S. Roukos, T. Ward, and W. J. Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proc. of ACL02, pages 311–318. find our card
  21. A. Stolcke. 2002. SRILM - an extensible language modeling toolkit. In Proc. of ICSLP02, pages 901–904. find
  22. D. Vickrey, L. Biewald, M. Teyssier, and D. Koller. 2005. Word-sense disambiguation for machine translation. In Proc. of EMNLP05, pages 771–778. find
  23. D. Wu. 1996. A polynomial-time algorithm for statistical machine translation. In Proc. of ACL96, pages 152–158. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/chan-2007-word-sense-disambiguation-improves-statistical/