Elamur Research⌕
☰
home / papers / 2004 / Scaling Web-based Acquisition of Entailment Relations

Scaling Web-based Acquisition of Entailment Relations

Paper · 2004 · Idan Szpektor, Hristo Tanev, Ido Dagan et al.

Summary

The paper tackles the problem of building a broad-coverage knowledge base of entailment relations. These are directional links between expressions, such as acquiring something implying owning it. Earlier Web-based paraphrase learners needed a hand-supplied seed anchor set for every target meaning, so they could not scale. The authors propose TE/ASE, a fully unsupervised method whose only input is a phrasal lexicon. For each lexicon entry (a "pivot") it works in two phases:

  • ASE builds a small parsed Web sample (MINIPAR dependency parses) and selects promising anchor sets.
  • TE extracts dependency-tree templates from Web sentences containing those anchors.

On 53 verbs judged by three annotators, the system reached about 44% precision with roughly 5.5 correct templates per verb. It found correct templates for 86% of the verbs. Allowing more anchor sets per pivot clearly raised both yield and precision. The results suggest that noisy Web data can match the precision of a clean news corpus while requiring far less input. This points toward a full-scale entailment KB built from an ordinary lexicon.

Key points

  • The only input is a lexicon. Pivot templates for verbs get subject and object/modifier slots, and sentences are parsed with MINIPAR (Lin, 1998).
  • ASE has four steps:
    • Retrieve a Web sample for the pivot.
    • Expand the sample with associated noun phrases ranked by tf·idf, discarding overly common Web phrases above the MAXPHRASEF threshold.
    • Extract slot and context anchors.
    • Filter candidate anchor sets by absolute frequency and conditional pivot probability.
  • Evaluation covered 53 verbs, 3 judges and 752 templates:
    • Average precision was 44.15% (42.39% under low majority), with an average yield of 5.5 correct templates per verb.
    • Kappa ranged from 0.55 to 0.63, and all three judges agreed on 69% of templates.
  • Lin and Pantel (2001) reported a yield of 10.1 and 50.3% precision on 9 templates from news data. The authors offer this as an informal reference point only.
  • In the main run, each verb was capped at 30 anchor sets. Raising the cap to 50 for 10 less frequent verbs increased yield from 3 to 6.5 and precision from 45.19% to 59.95%.

Interesting ideas

  • I1. Seeding from a lexicon instead of hand-made anchors. Each dictionary entry becomes a starting point for acquisition. This removes per-meaning manual seeding and could suit low-resource settings, such as building Ukrainian paraphrase or entailment resources from an existing word list.
  • I2. Choosing anchors before processing a whole corpus. Anchor sets are scored up front from a small sample, with filters that balance specificity against frequency. This makes Web or API querying affordable and fits any bootstrapping pipeline with a limited query budget.
  • I3. Topic-steering the sample with associated phrases. Strongly collocated noun phrases (tf·idf ranking, very common words excluded) are used to pull in topical rather than incidental usages of a pivot. The same trick can focus any search-based corpus collection on one word sense or domain.
  • I4. Templates as dependency subtrees, not fixed path shapes. The method aims for the most general structure the data supports, rather than fixed two-anchor paths. This lets it capture multiword and less trivial relations, which matters for languages with flexible word order.
  • I5. Reporting yield alongside precision, with multi-judge agreement. Counting correct items per input entry, and measuring how yield grows with search budget, gives a practical scalability estimate for resource-building projects.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Bar-Ilan UniversityRamat Gan · Israel118 works · 21,077 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

Cite

Cite as Idan Szpektor, Hristo Tanev, Ido Dagan and Bonaventura Coppola. 2004. Scaling Web-based Acquisition of Entailment Relations. Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{szpektor2004scaling,
  title = {{Scaling Web-based Acquisition of Entailment Relations}},
  author = {Idan Szpektor, Hristo Tanev, Ido Dagan and Bonaventura Coppola},
  booktitle = {Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing},
  year = {2004},
  url = {https://aclanthology.org/W04-3206/}
}

References

16 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Regina Barzilay and Lillian Lee. 2003. Learning to paraphrase: An unsupervised approach using multiple-sequence alignment. In Proceedings of HLT-NAACL 2003, pages 16–23, Edmonton, Canada.
  2. Regina Barzilay and Kathleen R. McKeown. 2001. Extracting paraphrases from a parallel corpus. In Proceedings of ACL 2001, pages 50–57, Toulose, France.
  3. Ido Dagan and Oren Glickman. 2004. Probabilistic textual entailment: Generic applied modeling of language variability. In PASCAL Workshop on Learning Methods for Text Understanding and Mining, Grenoble. find
  4. Florence Duclaye, François Yvon, and Olivier Collin. 2002. Using the Web as a linguistic resource for learning reformulations automatically. In Proceedings of LREC 2002, pages 390–396, Las Palmas, Spain. find
  5. Oren Glickman and Ido Dagan. 2003. Identifying lexical paraphrases from a single corpus: a case study for verbs. In Proceedings of RANLP 2003. find
  6. Ulf Hermjakob, Abdessamad Echihabi, and Daniel Marcu. 2003. Natural language based reformulation resource and Web Exploitation. In Ellen M. Voorhees and Lori P. Buckland, editors, Proceedings of the 11th Text Retrieval Conference (TREC 2002), Gaithersburg, MD. NIST. find
  7. Christian Jacquemin. 1999. Syntagmatic and paradigmatic representations of term variation. In Proceedings of ACL 1999, pages 341–348. find
  8. Dekang Lin and Patrick Pantel. 2001. Discovery of inference rules for Question Answering. Natural Language Engineering, 7(4):343–360. find
  9. Dekang Lin. 1998. Dependency-based evaluation of MINIPAR. In Proceedings of the Workshop on Evaluation of Parsing Systems at LREC 1998, Granada, Spain. find
  10. Dan Moldovan and Vasile Rus. 2001. Logic form transformation of WordNet and its applicability to Question Answering. In Proceedings of ACL 2001, pages 394–401, Toulose, France. find
  11. Bo Pang, Kevin Knight, and Daniel Marcu. 2003. Syntax-based alignment of multiple translations: Extracting paraphrases and generating new sentences. In Proceedings of HLT-NAACL 2003, Edmonton, Canada.
  12. Deepak Ravichandran and Eduard Hovy. 2002. Learning surface text patterns for a Question Answering system. In Proceedings of ACL 2002, Philadelphia, PA.
  13. Ellen Riloff and Rosie Jones. 1999. Learning dictionaries for Information Extraction by multi-level bootstrapping. In Proceedings of the Sixteenth National Conference on Artificial Intelligence (AAAI-99), pages 474–479. find
  14. Yusuke Shinyama, Satoshi Sekine, Kiyoshi Sudo, and Ralph Grishman. 2002. Automatic paraphrase acquisition from news articles. In Proceedings of Human Language Technology Conference (HLT 2002), San Diego, USA. find
  15. Kiyoshi Sudo, Satoshi Sekine, and Ralph Grishman. 2003. An improved extraction pattern representation model for automatic IE pattern acquisition. In Proceedings of ACL 2003. find
  16. Roman Yangarber, Ralph Grishman, Pasi Tapanainen, and Silja Huttunen. 2000. Unsupervised discovery of scenario-level patterns for Information Extraction. In Proceedings of COLING 2000. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/szpektor-2004-scaling-web-based-acquisition-entailment/