Elamur Research⌕
☰
home / papers / 2008 / Soft Syntactic Constraints for Hierarchical Phrased-Based Translation

Soft Syntactic Constraints for Hierarchical Phrased-Based Translation

Summary

The paper addresses how to bring linguistic syntax into statistical MT without losing the useful, linguistically "unmotivated" mappings that a model learns from parallel data. Most earlier work starts from a parse-based system and loosens it. Marton and Resnik go the other way. They start from Hiero, a hierarchical phrase-based model whose synchronous CFG is learned purely from word-aligned text, and add soft source-side constraints derived from parser output.

Chiang (2005) had tried a single binary feature that rewarded rules whose source span equals any constituent, and it gave no significant gain. The authors argue this feature was too coarse. They replace it with label-specific features (NP, VP, IP, CP, etc.) that separate two cases: matching a constituent, which earns a reward, and crossing its boundary, which incurs a penalty. Minimum error rate training (MERT) sets the weights, so the data decides how much to trust each constraint.

Experiments on Chinese-English and Arabic-English show substantial BLEU improvements over the baseline and over the undifferentiated feature. This revives an idea that had been abandoned.

Key points

  • The baseline is Hiero (Chiang 2005, 2007). It is a weighted synchronous CFG with nonterminals X and S, plus glue rules. Its log-linear features are:

    • phrase translation probabilities in both directions,
    • lexical weights in both directions,
    • a rule penalty.

    The feature weights are tuned with MERT.

  • The new feature space is the cross product of source constituent labels and {=, +}:

    • "=" fires when the rule's source side spans exactly one constituent and adds its weight to the score.
    • "+" fires when the source side crosses that constituent's boundary and subtracts its weight.

  • The authors define three variant families:

    • tied-weight features (e.g. NP_), where one weight rewards a match and penalizes a crossing;
    • two-feature models (e.g. NP2), with separate weights for match and crossing;
    • a "standard" set of ten labels, {IP, CP, VP, NP, PP, QP, ADVP, ADJP, DNP, LCP}, which leaves out labels such as PRN and FRAG.

  • The best constraints differ by language:

    • AdvP was the top performer for Arabic but is useless for Chinese, where AdvP constituents in the parses are almost always single words.
    • IP and VP variants helped for both languages.
    • Surprisingly, no NP variant gave much gain for Arabic.

  • Parse quality was ruled out as the main reason Chiang's original feature failed. The authors attribute the failure to the feature's lack of granularity.

Interesting ideas

  • I1. Start from a data-driven model and add syntax gently. The authors inject linguistic knowledge as optional scoring features instead of loosening a rigid syntax-based system. This design suits any setting where annotations are noisy or only partly relevant, such as low-resource parsing used for downstream tasks.
  • I2. Score rewards and violations separately. Giving "agrees with the structure" and "breaks the structure" their own weights captures more than a single bonus does. The same asymmetric reward/penalty design could apply to segmentation, chunking, or span-based extraction, wherever boundary violations matter.
  • I3. Make constraints type-specific. One constraint per label lets the model learn which categories transfer well between languages and which ones the parser handles reliably. This is a cheap way to expose which parts of an annotation scheme are actually useful for a given language pair.
  • I4. Let tuning decide how much to trust linguistics. Because MERT fits the feature weights, an unhelpful constraint can be driven toward zero instead of harming output. Any pipeline mixing heuristic or linguistic signals with learned scores can use this as a safety valve.
  • I5. Compare across languages as a diagnostic. Running the same feature set for two source languages showed that clause and verb-phrase constraints generalize, while others (AdvP, NP) depend on the language or the parser. The same evaluation design would help check which syntactic cues carry over to new languages, for example Ukrainian.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Machine translation · Natural language processing

Cited by

61 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 60 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{marton2008soft,
  title = {{Soft Syntactic Constraints for Hierarchical Phrased-Based Translation}},
  author = {Marton, Yuval and Resnik, Philip},
  booktitle = {Proceedings of ACL-08: HLT},
  year = {2008},
  url = {https://aclanthology.org/P08-1114/}
}

References

32 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alexandra Birch, Miles Osborne, and Philipp Koehn. 2007. CCG supertags in factored statistical machine translation. In Proceedings of the ACL Workshop on Statistical Machine Translation 2007. find
  2. P.F. Brown, S.A.D. Pietra, V.J.D. Pietra, and R.L. Mercer. 1993. The mathematics of statistical machine translation. Computational Linguistics, 19(2):263–313. find
  3. S. F. Chen and J. Goodman. 1998. An empirical study of smoothing techniques for language modeling. Tech. Report TR-10-98, Comp. Sci. Group, Harvard U. ↗ our card
  4. David Chiang. 2005. A hierarchical phrase-based model for statistical machine translation. In Proceedings of ACL-05, pages 263–270. find our card
  5. David Chiang. 2007. Hierarchical phrase-based translation. Computational Linguistics, 33(2):201–228. ↗ our card
  6. Michael Collins, Philipp Koehn, and Ivona Kucerova. 2005. Clause restructuring for statistical machine translation. In Proceedings of ACL-05. find our card
  7. Brooke Cowan, Ivona Kucerova, and Michael Collins. 2006. A discriminative model for tree-to-tree translation. In Proc. EMNLP. find
  8. S DeNeefe, K. Knight, W. Wang, and D. Marcu. 2007. What can syntax-based MT learn from phrase-based MT? In Proceedings of EMNLP-CoNLL. find
  9. J. Eisner. 2003. Learning non-isomorphic tree mappings for machine translation. In ACL Companion Vol. find our card
  10. Heidi Fox. 2002. Phrasal cohesion and statistical machine translation. In Proc. EMNLP 2002. find
  11. Michel Galley, Jonathan Graehl, Kevin Knight, Daniel Marcu, Steve DeNeefe, Wei Wang, and Ignacio Thayer. 2006. Scalable inference and training of context-rich syntactic translation models. In Proceedings of COLING/ACL-06. find our card
  12. H. Hassan, K. Sima’an, and A. Way. 2007. Integrating supertags into phrase-based statistical machine translation. In Proc. ACL-07, pages 288–295. find
  13. Zhongqiang Huang, Denis Filimonov, and Mary Harper. 2008. Accuracy enhancements for mandarin parsing. Tech. report, University of Maryland. find
  14. Dan Klein and Christopher D. Manning. 2003a. Accurate unlexicalized parsing. In Proceedings of ACL-03, pages 423–430. find our card
  15. Dan Klein and Christopher D. Manning. 2003b. Fast exact inference with a factored model for natural language parsing. Advances in Neural Information Processing Systems, 15(NIPS 2002):3–10. find
  16. Philipp Koehn and Hieu Hoang. 2007. Factored translation models. In Proc. EMNLP+CoNLL, pages 868– 876, Prague. find our card
  17. Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Proceedings of HLT-NAACL, pages 127–133. ↗
  18. Philipp Koehn. 2004. Statistical significance tests for machine translation evaluation. In Proc. EMNLP. find
  19. Adam Lopez. (to appear). Statistical machine translation. ACM Computing Surveys. Earlier version: A Survey of Statistical Machine Translation. U. of Maryland, UMIACS tech. report 2006-47. Apr 2007. find
  20. Daniel Marcu, Wei Wang, Abdessamad Echihabi, and Kevin Knight. 2006. SPMT: Statistical machine translation with syntactified target language phrases. In Proc. EMNLP, pages 44–52. find
  21. Franz Josef Och and Hermann Ney. 2000. Improved statistical alignment models. In Proceedings of the 38th Annual Meeting of the ACL, pages 440–447. GIZA++. find
  22. Franz Josef Och. 2003. Minimum error rate training in statistical machine translation. In Proceedings of the 41st Annual Meeting of the ACL, pages 160–167. find
  23. K. Owczarzak, B. Mellebeek, D. Groves, J. Van Genabith, and A. Way. 2006. Wrapper syntax for example-based machine translation. In Proceedings of the 7th Conference of the Association for Machine Translation in the Americas, pages 148–155. find
  24. Kishore Papineni, Salim Roukos, Todd Ward, John Henderson, and Florence Reeder. 2002. Corpusbased comprehensive and diagnostic MT evaluation: Initial Arabic, Chinese, French, and Spanish results. In Proceedings of the Human Language Technology Conference (ACL’2002), pages 124–127, San Diego, CA. find
  25. Stefan Riezler and John Maxwell. 2006. Grammatical machine translation. In Proc. HLT-NAACL, New York, NY. find
  26. Hendra Setiawan, Min-Yen Kan, and Haizhou Li. 2007. Ordering phrases with function words. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 712–719. find
  27. Andreas Stolcke. 2002. SRILM – an extensible language modeling toolkit. In Proceedings of the International Conference on Spoken Language Processing, volume 2, pages 901–904. find
  28. Huihsin Tseng, Pichuan Chang, Galen Andrew, Daniel Jurafsky, and Christopher Manning. 2005. A conditional random field word segmenter. In Fourth SIGHAN Workshop on Chinese Language Processing. find
  29. Chao Wang, Michael Collins, and Phillip Koehn. 2007a. Chinese syntactic reordering for statistical machine translation. In Proceedings of EMNLP. find our card
  30. Wei Wang, Kevin Knight, and Daniel Marcu. 2007b. Binarizing syntax trees to improve syntax-based machine translation accuracy. In Proc. EMNLP+CoNLL 2007. find
  31. Dekai Wu. 1997. Stochastic inversion transduction grammars and bilingual parsing of parallel corpora. Computational Linguistics, 23:377–404. find our card
  32. Andreas Zollmann and Ashish Venugopal. 2006. Syntax augmented machine translation via chart parsing. In Proceedings of the SMT Workshop, HLT-NAACL. ↗ our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/marton-2008-soft-syntactic-constraints-hierarchical-phrased/