Elamur Research⌕
☰
home / papers / 2005 / A Discriminative Matching Approach to Word Alignment

A Discriminative Matching Approach to Word Alignment

Summary

The paper reframes word alignment between parallel sentences as a learned, feature-rich bipartite matching task, as an alternative to generative IBM-style models trained with EM. Each candidate pair of words gets a score that is a linear function of arbitrary features: Dice association counts from unlabeled text, positional distortion, orthographic similarity, frequency and others. The alignment is then the best-scoring one-to-one matching. The weights are trained with a large-margin objective. Because loss-augmented matching can be written as a linear program with integral solutions, training becomes a saddle-point problem, which the authors solve with the extragradient method and projections computed via min-cost quadratic flow. On the English-French Hansards data, 100 hand-aligned training sentences and cheap count-based features give AER close to unsymmetrized IBM Model 4. Adding Model 4 outputs as features yields 5.4 AER, which beats intersected Model 4 by 22% relative. The approach brings flexible feature engineering to alignment, needs very little supervision, and trains much faster than GIZA++.

Key points

  • Data: the English-French Hansards set from the 2003 NAACL shared task, with 1.1M sentence pairs, a 39-pair validation set and a 447-sentence hand-aligned test set. The test set was split into 100 training and 347 evaluation sentences, so comparisons with other published results are not perfectly clean.
  • AER improved as feature groups were added:
    • Dice only: 29.8
    • plus distance features: 15.5
    • plus word shape and frequency: 14.4
    • plus common-word bilexical and next-pair Dice features: 10.7
    • plus Model 4 predictions: 5.4
  • Baselines for comparison: IBM Model 4 scored 8.9 and 9.7 in each direction and 6.9 when intersected. Raw Dice without matching scored about 36–39.
  • With the same features, an averaged perceptron reached only 8.1 AER. This is worse than the single intersected Model 4 feature.
  • Speed: GIZA++ needed 18 hours on the full corpus. The discriminative learner trained in 6 minutes on 100 sentences and in about 3 hours on 5K sentences labeled noisily by Model 4.

Interesting ideas

  • I1. Asymmetric Hamming loss as the training cost. Missed links (false negatives) were penalized 3 times more than spurious links, with the ratio tuned on validation data. This is a simple way to push a structured model toward the recall/precision balance that the evaluation metric rewards.
  • I2. Exact training via LP integrality. Since the matching polytope has integral vertices, loss-augmented inference can be relaxed to an LP without any approximation. The same trick applies to other structured outputs whose polytopes behave this way.
  • I3. Context dependencies moved into features. The matcher itself treats each link independently, but features such as the Dice score of the following word pair bring in neighbor information. This keeps inference cheap while still capturing some context.
  • I4. Stacking strong systems as features. Directional and intersected Model 4 predictions were fed in as edge features. A small labeled set was then enough to learn how much to trust each one, which is a cheap way to improve on existing aligners for a new language pair such as Ukrainian–English.
  • I5. Noisy automatic labels for scaling. Model 4 alignments served as extra training data when gold labels were scarce. This is a reusable recipe for bootstrapping supervised models from the output of a slower unsupervised system.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation · Natural language processing

Cited by

70 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 67 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{taskar2005discriminative,
  title = {{A Discriminative Matching Approach to Word Alignment}},
  author = {Taskar, Ben and Lacoste-Julien, Simon and Klein, Dan},
  booktitle = {Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing},
  year = {2005},
  doi = {10.3115/1220575.1220585},
  url = {https://doi.org/10.3115/1220575.1220585}
}

References

18 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. D. P. Bertsekas, L. C. Polymenakos, and P. Tseng. 1997. An e-relaxation method for separable convex cost network flow problems. SIAM J. Optim., 7(3):853–870. find
  2. P. F. Brown, J. Cocke, S. A. Della Pietra, V. J. Della Pietra, F. Jelinek, J. D. Lafferty, R. L. Mercer, and P. S. Roossin. 1990. A statistical approach to machine translation. Computational Linguistics, 16(2):79–85. find
  3. M. Collins. 2002. Discriminative training methods for hidden markov models: Theory and experiments with perceptron algorithms. In Proc. EMNLP. find our card
  4. T. H. Cormen, C. E. Leiserson, and R. L. Rivest. 1990. Introduction to Algorithms. MIT Press, Cambridge, MA. find
  5. L. R. Dice. 1945. Measures of the amount of ecologic association between species. Journal of Ecology, 26:297– 302. find
  6. F. Guerriero and P. Tseng. 2002. Implementation and test of auction methods for solving generalized network flow problems with separable convex cost. Journal of Optimization Theory and Applications, 115(1):113–144, October. find
  7. B.S. He and L. Z. Liao. 2002. Improvements of some projection methods for monotone nonlinear variational inequalities. JOTA, 112:111:128. find
  8. G. M. Korpelevich. 1976. The extragradient method for finding saddle points and other problems. Ekonomika i Matematicheskie Metody, 12:747:756. find
  9. E. Matusov, R. Zens, and H. Ney. 2004. Symmetric word alignments for statistical machine translation. In Proc. of COLING 2004. find
  10. I. D. Melamed. 2000. Models of translational equivalence among words. Computational Linguistics, 26(2):221– 249. find
  11. R. Mihalcea and T. Pedersen. 2003. An evaluation exercise for word alignment. In Proceedings of the HLT-NAACL 2003 Workshop, Building and Using parallel Texts: Data Driven Machine Translation and Beyond, pages 1–6, Edmonton, Alberta, Canada. find our card
  12. F. Och and H. Ney. 2003. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19–52. find
  13. A. Schrijver. 2003. Combinatorial Optimization: Polyhedra and Efficiency. Springer. find
  14. B. Taskar, V. Chatalbashev, D. Koller, and C. Guestrin. 2005a. Learning structured prediction models: a large margin approach. In Proceedings of the International Conference on Machine Learning. find
  15. B. Taskar, S. Lacoste-Julien, and M. Jordan. 2005b. Structured prediction via the extragradient method. In Proceedings of Neural Information Processing Systems. find
  16. J. Tiedemann. 2003. Combining clues for word alignment. In Proceedings of EACL. find
  17. L. G. Valiant. 1979. The complexity of computing the permanent. Theoretical Computer Science, 8:189–201. find
  18. S. Vogel, H. Ney, and C. Tillmann. 1996. HMM-based word alignment in statistical translation. In COLING 16, pages 836–841. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/taskar-2005-discriminative-matching-approach-word-alignment/