Elamur Research⌕
☰
home / papers / 2014 / Back to Basics for Monolingual Alignment: Exploiting Word Similarity and Contextual Evidence

Back to Basics for Monolingual Alignment: Exploiting Word Similarity and Contextual Evidence

Paper · 2014 · Md Arafat Sultan, Steven Bethard and Tamara Sumner

Summary

The paper tackles monolingual alignment, which means finding which words in two English sentences correspond in meaning. This step supports entailment recognition, semantic similarity, paraphrase detection and QA. Earlier aligners such as MANLI and JacanaAlign relied heavily on supervised training with the MSR alignment corpus. The authors instead build a lightweight, almost unsupervised word aligner on two basic ideas: aligned words should be semantically similar, and their surrounding contexts should also correspond.

Word similarity has three levels: an exact word or lemma match, a pair listed in PPDB, or no similarity. Contextual evidence comes from two sources:

  • Stanford typed dependencies, extended with a small hand-built mapping of equivalent dependency types (e.g., dobj ≈ rcmod).
  • Nearby words in the surface text.

A greedy pipeline aligns word pairs in a fixed order: identical sequences, then named entities, then content words, then stop words.

The aligner reaches 88–92% F1 on two alignment benchmarks, beating the previous best system. It also transfers well to paraphrase detection and textual similarity. Because it is simple and its output does not depend on sentence order, it is easy to replicate and reuse.

Key points

  • On the MSR alignment corpus (Brockett, 2007), the aligner scores 91.7% F1. This is 3.1 points absolute above JacanaAlign (Yao et al., 2013a), or 27.2% fewer errors. It also comes out ahead on the Thadani et al. (2012) dataset.
  • PPDB, using its largest lexical package (XXXL), is the only word-similarity resource. Preprocessing uses the lemmatizer, POS tagger, NER and dependency parser from Stanford CoreNLP.
  • On the MSR paraphrase corpus (4076 dev / 1725 test pairs), a simple thresholded alignment score gives 73.4% accuracy and 81.2% F1. This beats other aligners but stays below the best paraphrase system (Madnani et al., 2012, 77.4% accuracy).
  • Known limitations: it cannot align phrases or multiword expressions, it ignores long-distance links such as coreference, and it misses pairs that PPDB does not cover.
  • The code is released at github.com/ma-sultan/monolingual-word-aligner.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Colorado BoulderBoulder · United States52 works · 49,647 citations🎓University of Alabama at BirminghamBirmingham · United States5 works · 9,720 citations🎓University of Colorado SystemBoulder · United States7 works · 1,348 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

Cite

Cite as Md Arafat Sultan, Steven Bethard and Tamara Sumner. 2014. Back to Basics for Monolingual Alignment: Exploiting Word Similarity and Contextual Evidence. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl_a_00178

@article{sultan2014back,
  title = {{Back to Basics for Monolingual Alignment: Exploiting Word Similarity and Contextual Evidence}},
  author = {Md Arafat Sultan, Steven Bethard and Tamara Sumner},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2014},
  doi = {10.1162/tacl\_a\_00178},
  url = {https://doi.org/10.1162/tacl_a_00178}
}

References

24 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Eneko Agirre, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, and Weiwei Guo. 2013. *SEM 2013 shared task: Semantic Textual Similarity. In Proceedings of the Second Joint Conference on Lexical and Computational Semantics. Association for Computational Linguistics, 32-43.
  2. Colin Bannard and Chris Callison-Burch. 2005. Paraphrasing with Bilingual Parallel Corpora. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics. Association for Computational Linguistics, 597-604. Daniel Bär, Chris Biemann, Iryna Gurevych, and Torsten Zesch. 2012.…
  3. Roy Bar-Haim, Ido Dagan, Bill Dolan, Lisa Ferro, Danilo Giampiccolo, Bernardo Magnini, and Idan Szpektor. 2006. The Second PASCAL Recognising Textual Entailment Challenge. In Proceedings of The Second PASCAL Recognising Textual Entailment Challenge. find
  4. Chris Brockett. 2007. Aligning the RTE 2006 Corpus. Technical Report MSR-TR-2007-77, Microsoft Research. find
  5. Nathanael Chambers, Daniel Cer, Trond Grenager, David Hall, Chloe Kiddon, Bill MacCartney, Marie-Catherine de Marneffe, Daniel Ramage, Eric Yeh, and Christopher D Manning. 2007. Learning alignments and leveraging natural logic. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Para… find
  6. Ming-Wei Chang, Dan Goldwasser, Dan Roth, and Vivek Srikumar. 2010. Discriminative Learning over Constrained Latent Representations. In Proceedings of the 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics Association for Computational Linguistics,… find
  7. Dipanjan Das and Noah A. Smith. 2009. Paraphrase Identication as Probabilistic Quasi-Synchronous Recognition. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. Association for Computati… find
  8. Marie-Catherine de Marneffe, Bill MacCartney, and Christopher D. Manning. 2006. Generating Typed Dependency Parses from Phrase Structure Parses. In Proceedings of the International Conference on Language Resources and Evaluation. 449-454.
  9. Marie-Catherine de Marneffe and Christopher D. Manning. 2008. Stanford typed dependencies manual. find
  10. Technical Report, Stanford University. Bill Dolan, Chris Quirk, and Chris Brockett. 2004. Unsupervised Construction of Large Paraphrase Corpora: Exploiting Massively Parallel News Sources. In Proceedings of the International Conference on Computational Linguistics. Association for Computational Ling…
  11. Jenny Rose Finkel, Trond Grenager, and Christopher Manning. 2005. Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 363-370.
  12. Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch. 2013. PPDB: The Paraphrase Database. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 758-764.
  13. Lushan Han, Abhay Kashyap, Tim Finin, James Mayeld, and Jonathan Weese. 2013. UMBC EBIQUITY-CORE: Semantic Textual Similarity Systems. In Proceedings of the Second Joint Conference on Lexical and Computational Semantics, Volume 1. Association for Computational Linguistics, 44-52.
  14. Andrew Hickl and Jeremy Bensley. 2007. A Discourse Commitment-Based Framework for Recognizing Textual Entailment. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing. Association for Computational Linguistics, 171-176. find
  15. Andrew Hickl, Jeremy Bensley, John Williams, Kirk Roberts, Bryan Rink, and Ying Shi. 2006. Recognizing Textual Entailment with LCCs GROUNDHOG System. In Proceedings of the Second PASCAL Challenges Workshop on Recognizing Textual Entailment. find
  16. Milen Kouylekov and Bernardo Magnini. 2005. Recognizing textual entailment with tree edit distance algorithms. In Proceedings of the PASCAL Challenges Workshop: Recognising Textual Entailment Challenge 17-20. find
  17. Bill MacCartney, Michel Galley, and Christopher D. Manning. 2008. A Phrase-Based Alignment Model for Natural Language Inference. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 802-811. find
  18. Nitin Madnani, Joel Tetreault, and Martin Chodorow. 2012. Re-examining Machine Translation Metrics for Paraphrase Identification. In Proceedings of 2012 Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 182-190.
  19. Kapil Thadani and Kathleen McKeown. 2011. Optimal and Syntactically-Informed Decoding for Monolingual Phrase-Based Alignment. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 254-259. find
  20. Kapil Thadani, Scott Martin, and Michael White. 2012. A Joint Phrasal and Dependency Model for Paraphrase Alignment. In Proceedings of COLING 2012: Posters. 1229-1238. find
  21. Kristina Toutanova, Dan Klein, Christopher D. Manning, and Yoram Singer. 2003 Feature-rich Part-of-speech Tagging with a Cyclic Dependency Network In Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics. Associati…
  22. Xuchen Yao, Benjamin Van Durme, Chris Callison-Burch, and Peter Clark. 2013a. A Lightweight and High Performance Monolingual Word Aligner. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 702-707. find
  23. Xuchen Yao, Benjamin Van Durme, Chris Callison-Burch, and Peter Clark. 2013b. Semi-Markov Phrase-based Monolingual Alignment. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 590-600. find
  24. Xuchen Yao, Benjamin Van Durme, Chris Callison-Burch, and Peter Clark. 2013c. Answer Extraction as Sequence Tagging with Tree Edit Distance. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sultan-2014-back-basics-monolingual-alignment-exploiting/