Elamur Research
☰
home / papers / 2010 / Learning to Translate with Source and Target Syntax

Learning to Translate with Source and Target Syntax

What it is about

Tree-to-tree translation, with syntax on both sides, has often done worse than phrase-based systems. The paper studies why, and proposes a simpler method. Instead of forcing the two parse trees to match, it keeps the hierarchical phrase-based model (Hiero) and adds syntactic features to it: whether a span's source side and target side match syntactic categories, and how well the chosen structure fits both grammars. The grammar is extracted with "fuzzy" tree-to-tree rules, which allow many new non-terminal labels, and the features let the model learn when to rely on syntax and when to fall back to monotone phrase-based translation. On Chinese-English, the fuzzy extraction gives about 0.9 BLEU on the test set over a string-to-string baseline, with an exact tree-to-tree extraction below it. On Arabic-English, the gain is smaller. The paper also reports that the model learns more well-formed structures, and it shows the frequency of each rewrite on both sides.

Ideas

For Mova

The code is tools/translate/src/profile.rs and tools/translate/src/meta.rs. The method is count-based features over a hierarchical model, which fits Mova. Its neural parts are absent, and its gains are small, so each feature must be judged by the residual of exact mode, not by BLEU.

Library summary

The Research library's own summary of the paper, in our words — not the paper's text.

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/P10-1146.pdf

Summary

Chiang tackles a long-standing puzzle in syntax-based statistical MT. Models that use parse trees on only one side, or none, work well, but tree-to-tree models that use both source and target syntax have usually lagged behind phrase-based systems. The paper identifies two causes. Exact extraction discards many useful phrase pairs because of parser errors and mismatched annotation conventions. Strict label matching at decoding time also overconstrains the search. The proposed remedy has two parts. First, a "fuzzy" tree-to-tree extraction gives every phrase a possibly complex SAMT-style category (product and slash categories), so no rules are lost compared with string-to-string extraction. Second, syntactic compatibility becomes a set of soft features inside Hiero rather than a hard requirement, with weights tuned by MIRA. On Chinese-English and Arabic-English newswire, this system beats both the non-syntactic Hiero baseline and exact tree-to-tree extraction. The model also learns by itself to build more syntactically well-formed derivations. The work shows that two-sided syntax helps when the data decides how strictly to enforce it.

Key points

  • Phrase loss under syntactic filters (Table 1): On 2000 Chinese-English sentences with human annotations, exact tree-to-tree extraction keeps only 68% of the maximum number of nested phrase pairs (60% with automatic parses and alignments). Tree-to-string and string-to-tree keep about 72–78%.
  • BLEU on Chinese-English:
    • Hiero baseline: 32.7/23.4 (dev/test).
    • Raising the distortion limit from 10 to 20: 33.3/23.7.
    • Fuzzy tree-to-tree: 33.9/24.3.
    • Exact tree-to-tree: 32.8/23.9.
  • BLEU on Arabic-English:
    • Baseline: 48.7/48.9.
    • Fuzzy tree-to-tree: 49.4/49.7.
    • Exact tree-to-tree: 46.6/47.5, which hurts badly.
  • Grammar size: Exact extraction shrinks the grammar sharply (440M → 50M rules for Chinese). A nesting heuristic cuts the fuzzy grammar by 2.4× (Chinese) and 4.2× (Arabic) with almost no change in BLEU.
  • Derivation behaviour (Table 4): With the syntax features, glue-rule use drops (e.g., 25% → 18% on the Chinese side) and matching substitutions increase (17% → 30%). Tuning used MIRA on about 3000 sentences, with 90% for optimisation and 10% held out to decide when to stop.

Interesting ideas

  • Turn hard constraints into learned features: Instead of forbidding label mismatches, the model scores them and lets tuning decide how much they matter. This is useful wherever linguistic annotation is noisy, for example soft UD-based constraints in a generation or parsing pipeline.
  • Never lose coverage by labelling every span: Compositional categories (concatenations, "X missing Y") ensure every span has some label. Adding syntax then only adds information and never removes rules.
  • Collapse rules that differ only in labels: Keeping only the most frequent labelling preserves a one-to-one mapping with the plain grammar. That makes it possible to compare a syntactic and a non-syntactic system derivation by derivation, a clean analysis design for any "with vs. without annotation" study.
  • Blame annotation conventions before blaming the formalism: Many failed extractions come from flat structures or uncertain attachment choices, not from deep cross-lingual divergence. This is worth checking before adopting more powerful formalisms, especially for lower-resource treebanks such as Ukrainian.
  • Inspect what the model learned, not just its score: Glue-rule frequency and match/mismatch rates gave interpretable evidence, such as better punctuation placement and article insertion in English NPs. These counts show the model actually exploited syntax beyond the BLEU gain.

Topics: Machine translation

Cite

The paper itself, not this card. BibTeX from the library's record of the work.

@inproceedings{chiang2010learning,
  title = {{Learning to Translate with Source and Target Syntax}},
  author = {Chiang, David},
  booktitle = {Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics},
  year = {2010},
  url = {https://aclanthology.org/P10-1146/}
}

References

27 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Vamshi Ambati and Alon Lavie. 2008. Improving syntax driven translation models by re-structuring divergent and non-isomorphic parse tree structures. In Proc. AMTA-2008 Student Research Workshop, pages 235–244. find
  2. Yehoshua Bar-Hillel. 1953. A quasi-arithmetical notation for syntactic description. Language, 29(1):47–58. find
  3. Stanley F. Chen and Joshua Goodman. 1998. An empirical study of smoothing techniques for language modeling. Technical Report TR-10-98, Harvard University Center for Research in Computing Technology. find
  4. David Chiang, Wei Wang, and Kevin Knight. 2009. 11,001 new features for statistical machine translation. In Proc. find
  5. David Chiang. 2005. A hierarchical phrase-based model for statistical machine translation. In Proc. ACL 2005, pages 263–270. find our card
  6. David Chiang. 2007. Hierarchical phrase-based translation. Computational Linguistics, 33(2):201–228. find our card
  7. Michael Collins. 1997. Three generative lexicalised models for statistical parsing. In Proc. ACL-EACL, pages 16–23. find
  8. Koby Crammer and Yoram Singer. 2003. Ultraconservative online algorithms for multiclass problems. Journal of Machine Learning Research, 3:951–991. find
  9. Jason Eisner. 2003. Learning non-isomorphic tree mappings for machine translation. In Proc. ACL 2003 Companion Volume, pages 205–208. find our card
  10. Victoria Fossum, Kevin Knight, and Steven Abney. 2008. Using syntax to improve word alignment for syntax-based statistical machine translation. In Proc. Third Workshop on Statistical Machine Translation, pages 44–52. find
  11. Alexander Fraser and Daniel Marcu. 2007. Getting the structure right for word alignment: LEAF. In Proc. EMNLP 2007, pages 51–60. find
  12. Michel Galley, Mark Hopkins, Kevin Knight, and Daniel Marcu. 2004. What’s in a translation rule? In Proc. HLT-NAACL 2004, pages 273–280. find our card
  13. Michel Galley, Jonathan Graehl, Kevin Knight, Daniel Marcu, Steve DeNeefe, Wei Wang, and Ignacio Thayer. 2006. Scalable inference and training of context-rich syntactic translation models. In Proc. COLING-ACL 2006, pages 961–968. find our card
  14. Mary Hearne and Andy Way. 2003. Seeing the wood for the trees: Data-Oriented Translation. In Proc. MT Summit IX, pages 165–172. find
  15. Liang Huang, Kevin Knight, and Aravind Joshi. 2006. Statistical syntax-directed translation with extended domain of locality. In Proc. AMTA 2006, pages 65–73. find our card
  16. Alon Lavie, Alok Parlikar, and Vamshi Ambati. 2008. Syntax-driven learning of sub-sentential translation equivalents and translation rules from parsed parallel corpora. In Proc. SSST-2, pages 87–95. find
  17. Yang Liu, Qun Liu, and Shouxun Lin. 2006. Tree-to-string alignment template for statistical machine translation. In Proc. COLING-ACL 2006, pages 609–616. find our card
  18. Yang Liu, Yajuan Lü, and Qun Liu. 2009. Improving tree-to-tree translation with packed forests. In Proc. ACL 2009, pages 558–566. find
  19. I. Dan Melamed, Giorgio Satta, and Ben Wellington. 2004. Generalized multitext grammars. In Proc. ACL 2004, pages 661–668. find
  20. Slav Petrov, Leon Barrett, Romain Thibaux, and Dan Klein. 2006. Learning accurate, compact, and interpretable tree annotation. In Proc. COLING-ACL 2006, pages 433–440. find our card
  21. Arjen Poutsma. 2000. Data-Oriented Translation. In Proc. COLING 2000, pages 635–641. find
  22. David Talbot and Thorsten Brants. 2008. Randomized language models via perfect hash functions. In Proc. ACL-08: HLT, pages 505–513. find
  23. Ashish Venugopal, Andreas Zollmann, Noah A. Smith, and Stephan Vogel. 2009. Preference grammars: Softening syntactic constraints to improve statistical machine translation. In Proc. find
  24. David J. Weir. 1988. Characterizing Mildly Context-Sensitive Grammar Formalisms. Ph.D. thesis, University of Pennsylvania. find
  25. Benjamin Wellington, Sonjia Waxmonsky, and I. Dan Melamed. 2006. Empirical lower bounds on the complexity of translational equivalence. In Proc. COLING-ACL 2006, pages 977–984. find
  26. Min Zhang, Hongfei Jiang, Aiti Aw, Haizhou Li, Chew Lim Tan, and Sheng Li. 2008. A tree sequence alignment-based tree-to-tree translation model. In Proc. ACL-08: HLT, pages 559–567. find
  27. Andreas Zollmann and Ashish Venugopal. 2006. Syntax augmented machine translation via chart parsing. In Proc. Workshop on Statistical Machine Translation, pages 138–141. find