Elamur Research⌕
☰
home / papers / 2015 / Multi-Level Alignments As An Extensible Representation Basis for Textual Entailment Algorithms

Multi-Level Alignments As An Extensible Representation Basis for Textual Entailment Algorithms

Paper · 2015 · Tae-Gil Noh, Sebastian Padó, Vered Shwartz et al.

Summary

Building Textual Entailment (TE) systems is expensive. Shared platforms already standardize preprocessing, annotation types and knowledge resources, but the algorithms themselves stay monolithic and hard to extend or port to new languages. The authors propose multi-level alignment as a common intermediate layer. Independent "alignment producers" add weighted, labeled links between any linguistic units of Text and Hypothesis, and these links may signal either match or mismatch. "Alignment consumers" then turn the pooled links into features for a supervised classifier. Unlike the earlier schema of Dagan et al., no step chooses a single best alignment, so evidence from different sources coexists. A pilot system, MultiAlign, was built inside the Excitement Open Platform with three aligners and four coverage features. On RTE-3 it matched or beat mature open-source engines (BIUTEE, TIE, EDITS) in English, German and Italian, and it performed similarly on real application data. The point is that a modular, language-independent design can reach state-of-the-art results with very little engineering.

Key points

  • The pilot uses three aligners:
    • a lexical aligner over lemmas, using WordNet and VerbOcean for EN, Italian WordNet for IT, and GermaNet plus DerivBase for DE;
    • a paraphrase aligner over token sequences, using METEOR paraphrase tables;
    • a resource-free lemma identity aligner, which is important for named entities.
  • The four features all measure how much of H is covered by alignments: all words, content words, verbs and proper names. A word counts as covered if any aligner linked it.
  • Accuracy on RTE-3 (800 training / 800 test pairs, translated into DE and IT):
    • EN: 67.0, tied with BIUTEE; TIE scored 65.2 and EDITS 63.6.
    • DE: 64.5, against 63.1 for TIE.
    • IT: 65.4, against 62.6 for EDITS.
  • F1 on application data (EN 5300, IT 1700, DE 1274 balanced pairs from entailment graphs and e-mail categorization):
    • EN: 69.2, against 71.3 for BIUTEE.
    • DE: 72.4, tied with TIE.
    • IT: 69.5, against 65.6 for EDITS.
  • The infrastructure combines TreeTagger pipelines, UIMA CAS with the DKPro type system, and a BRAT-based visualizer that displays each link with its source and score.

Interesting ideas

  • I1. Skip alignment selection and keep every aligner's links side by side, positive and negative, so that interactions are handled only at feature-extraction time; adding a new evidence source then never breaks existing ones.
  • I2. Split each module into an aligner plus its own feature extractors behind one shared data structure, which keeps even complex pipelines to a few clean interfaces.
  • I3. Measure coverage of the hypothesis separately per word class (content words, verbs, names), because an entailment can survive weak overall coverage as long as the crucial words are matched.
  • I4. Make feature code language-independent by working only over a shared annotation type system, so the same extractor runs unchanged for three languages.
  • I5. Use a visualizer that shows every link with its origin and weight; it exposed a bug where the noun "judges" was misread as a verb and linked via VerbOcean.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): The world builder could store competing entity or state matches from different modules as labeled, scored links and let the reasoning layer resolve conflicts explicitly instead of committing early.
  • I2 (answer): Readers for school tests or fable QA could each contribute alignment-style evidence between question and story graph, with answer scoring written as small, separable extractors.
  • I3 (answer): When checking whether a candidate answer follows from the text, report verb and entity coverage of the answer frame separately, which yields an explainable, non-bag-of-words justification.
  • I4 (back): Translation through the meta-world could check source and target meaning graphs with identical coverage features, since both live in one type system.
  • I5 (learn): Rendering cross-module links with their provenance would help detect systematic upstream errors, such as tagging or lemma mistakes, and turn them into repair rules or negative controls.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Bar-Ilan UniversityRamat Gan · Israel118 works · 21,077 citations🎓University of StuttgartStuttgart · Germany75 works · 46,113 citations🔬German Research Centre for Artificial IntelligenceKaiserslautern · Germany42 works · 28,596 citations🎓Heidelberg UniversityHeidelberg · Germany27 works · 40,944 citations

📚 Shelves 4 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works

🧭 Topics 1 Reasoning · 510 works

Cite

Cite as Tae-Gil Noh, Sebastian Padó, Vered Shwartz, Ido Dagan, Vivi Nastase, Kathrin Eichler, Lili Kotlerman and Meni Adler. 2015. Multi-Level Alignments As An Extensible Representation Basis for Textual Entailment Algorithms. Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/s15-1022

@inproceedings{noh2015multi,
  title = {{Multi-Level Alignments As An Extensible Representation Basis for Textual Entailment Algorithms}},
  author = {Tae-Gil Noh, Sebastian Padó, Vered Shwartz, Ido Dagan, Vivi Nastase, Kathrin Eichler, Lili Kotlerman and Meni Adler},
  booktitle = {Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics},
  year = {2015},
  doi = {10.18653/v1/s15-1022},
  url = {https://doi.org/10.18653/v1/s15-1022}
}

References

21 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Colin Bannard and Chris Callison-Burch. 2005. Paraphrasing with bilingual parallel corpora. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 597–604, Ann Arbor, MI. our card
  2. Jonathan Berant, Ido Dagan, and Jacob Goldberger. 2012. Learning entailment relations by global graph structure optimization. Computational Linguistics, 38(1):73– 111. ↗
  3. Ido Dagan and Oren Glickman. 2004. Probabilistic textual entailment: Generic applied modeling of language variability. In PASCAL workshop on Learning Methods for Text Understanding and Mining, Grenoble, France. find
  4. Ido Dagan, Dan Roth, Mark Sammons, and Fabio Massimo Zanzotto. 2013. Recognizing Textual Entailment: Models and Applications. Number 17 in Synthesis Lectures on Human Language Technologies. Morgan & Claypool. ↗
  5. Richard Eckart de Castilho and Iryna Gurevych. 2014. A broad-coverage collection of portable NLP components for building shareable analysis pipelines. In Proceedings of the Workshop on Open Infrastructures and Analysis Frameworks for HLT at COLING 2014, pages 1–11, Dublin, Ireland, August. our card
  6. Michael Denkowski and Alon Lavie. 2014. Meteor universal: Language specific translation evaluation for any target language. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 376–380, Baltimore, MD. our card
  7. Georgiana Dinu and Rui Wang. 2009. Inference rules and their application to recognizing textual entailment. In Proceedings of the 12th Conference of the European Chapter of the ACL, pages 211–219, Athens, Greece. find
  8. Kathrin Eichler, Aleksandra Gabryszak, and Günter Neumann. 2014. An analysis of textual inference in German customer emails. In Proceedings of the Third Joint Conference on Lexical and Computational Semantics, pages 69–74, Dublin, Ireland. find
  9. David Ferrucci and Adam Lally. 2004. UIMA: An architectural approach to unstructured information processing in the corporate research environment. Natural Language Engineering, 10(3–4):327–348. ↗
  10. Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and Bill Dolan. 2007. The third PASCAL recognising textual entailment challenge. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pages 1–9, Prague, Czech Republic. find
  11. Sanda Harabagiu and Andrew Hickl. 2006. Methods for using textual entailment in open-domain question answering. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 905–912, Sydney, Australia. our card
  12. Lili Kotlerman, Ido Dagan, Bernardo Magnini, and Luisa Bentivogli. 2015. Textual entailment graphs. Natural Language Engineering. ↗
  13. Milen Kouylekov and Matteo Negri. 2010. An open-source package for recognizing textual entailment. In Proceedings of the ACL 2010 System Demonstrations, pages 42–47, Uppsala, Sweden. our card
  14. Bill MacCartney, Trond Grenager, Marie-Catherine de Marneffe, Daniel Cer, and Christopher D Manning. 2006. Learning to recognize features of valid textual entailments. In Proceedings of the Human Language Technology Conference of the NAACL, pages 41–48, New York City, USA. find
  15. Bill MacCartney, Michel Galley, and Christopher D Manning. 2008. A phrase-based alignment model for natural language inference. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 802–811, Honolulu, Hawaii. find
  16. Bernardo Magnini, Roberto Zanoli, Ido Dagan, Kathrin Eichler, Günter Neumann, Tae-Gil Noh, Sebastian Padó, Asher Stern, and Omer Levy. 2014. The EX-CITEMENT open platform for textual inferences. In Proceedings of the ACL 2014 System Demonstrations, pages 43–48, Baltimore, MD. find
  17. Rodney D Nielsen, Wayne Ward, and James H Martin. 2009. Recognizing entailment in intelligent tutoring systems. Journal of Natural Language Engineering, 15(4):479–501. ↗
  18. Sebastian Padó, Tae-Gil Noh, Asher Stern, Rui Wang, and Roberto Zanoli. 2015. Design and realization of a modular architecture for textual entailment. Journal of Natural Language Engineering, 21(2):167–200. find
  19. Asher Stern and Ido Dagan. 2012. BIUTEE: A modular open-source system for recognizing textual entailment. In Proceedings of the ACL 2012 System Demonstrations, pages 73–78, Jeju Island, Korea. our card
  20. Rui Wang and Yi Zhang. 2009. Recognizing textual relatedness with predicate-argument structures. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 784–792, Singapore. find
  21. Britta Zeller, Jan Šnajder, and Sebastian Padó. 2013. DErivBase: Inducing and evaluating a derivational morphology resource for German. In Proceedings of ACL, pages 1201–1211, Sofia, Bulgaria. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/noh-2015-multi-level-alignments-as-extensible/