Elamur Research⌕
☰
home / papers / 2005 / The Distributional Inclusion Hypotheses and Lexical Entailment

The Distributional Inclusion Hypotheses and Lexical Entailment

Summary

Classic distributional similarity tends to group words that are only loosely related (e.g., company–government). It does not tell us whether one word can stand in for another while keeping the meaning inferable, which is what QA, IE and paraphrase systems need. Geffet and Dagan propose two Distributional Inclusion Hypotheses as a refinement of Harris' idea. The first says that if v entails w, the characteristic syntactic contexts of v should also appear with w. The second states the converse: such inclusion suggests entailment. To test these claims, they built an Inclusion Testing Algorithm (ITA). It takes Minipar dependency features from a corpus and checks whether they also occur with the other word, using web queries to offset data sparseness. A manual, sense-level analysis found that the first hypothesis holds almost completely, while the second holds only partly. Used as a filter on RFF-based similarity lists, ITA raised precision from 0.52 to 0.70–0.74 with only a modest loss in recall. This shows that asymmetric feature inclusion is a more useful signal for lexical entailment than symmetric overlap.

Key points

  • The evaluation data comes from the authors' earlier work: 1,200 noun pairs, built from the top-40 RFF similarity neighbours of 30 random nouns. Annotators judged each pair for lexical entailment, reaching Kappa 0.75 and 0.83. For this paper the authors also added the direction of entailment.

  • RFF (Relative Feature Focus) re-weighting had already given about 10% higher precision than Lin's MI-based measure. Even so, over 47% of the top-40 pairs were not entailing. Fewer than 20% of the extracted entailments were found in WordNet.

  • The filter was tested on a 200-pair sample, with precision / recall / F1 as follows:

    MethodPrecisionRecallF1
    ITA-200.7000.8750.777
    ITA-400.7400.8460.789
    RFF-top-400.5201.0000.684
    RFF-top-260.5610.7010.624

  • ITA discarded about 60% of the incorrect pairs and about 35% of the output overall. Relative recall dropped by only 13%.

  • After sense-level analysis, 27.4% of non-entailing pairs still passed the inclusion test. The causes were:

    • generic domain features, as in valley–town;
    • anaphoric, context-dependent entailment;
    • very broad, ambiguous words such as element.

Interesting ideas

  • I1. Directional inclusion instead of symmetric similarity. The test asks whether one word's defining contexts are a subset of the other's contexts. This naturally captures asymmetric relations such as hyponymy, which cosine or Lin scores cannot express.
  • I2. The web as a sparseness oracle. Before treating a missing word–feature co-occurrence as real counter-evidence, the method checks it against a very large external source. This helps any pipeline built on small corpora, such as one for a lower-resource language.
  • I3. Sample features at random rather than taking the top-weighted ones. The top RFF features turned out to be shared across many similar words and were therefore less discriminative. In feature-based tests, the highest-weighted features are not always the most diagnostic.
  • I4. Compare against an equal-size cutoff. The authors set the filter against simply trimming the ranked list to the same size (top-26). This shows the gain comes from the filter itself, not just from producing shorter output.
  • I5. Check errors at the sense level. Re-examining failures by word sense separated real counterexamples from polysemy artifacts. This gives a fairer picture of whether a hypothesis actually holds.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

92 papers of our library cite it (of the 140884 whose bibliographies we read); 3 of them have a page here, 89 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/1219840.1219854

@inproceedings{geffet2005distributional,
  title = {{The Distributional Inclusion Hypotheses and Lexical Entailment}},
  author = {Geffet, Maayan and Dagan, Ido},
  booktitle = {Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05)},
  year = {2005},
  doi = {10.3115/1219840.1219854},
  url = {https://doi.org/10.3115/1219840.1219854}
}

References

20 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Chklovski, Timothy and Patrick Pantel. 2004. V ERB O CEAN : Mining the Web for Fine-Grained Semantic Verb Relations. In Proc. of EMNLP-04. Barcelona, Spain. find
  2. Church, Kenneth W. and Hanks Patrick. 1990. Word association norms, mutual information, and Lexicography. Computational Linguistics, 16(1), pp. 22–29. find our card
  3. Dagan, Ido. 2000. Contextual Word Similarity, in Rob Dale, Hermann Moisl and Harold Somers (Eds.), Handbook of Natural Language Processing, Marcel Dekker Inc, 2000, Chapter 19, pp. 459-476. find
  4. Dagan, Ido, Oren Glickman and Bernardo Magnini. 2005. The PASCAL Recognizing Textual Entailment Challenge. In Proc. of the PASCAL Challenges Workshop for Recognizing Textual Entailment. Southampton, U.K. find
  5. Geffet, Maayan and Ido Dagan, 2004. Feature Vector Quality and Distributional Similarity. In Proc. of Coling-04. Geneva. Switzerland. find
  6. Grefenstette, Gregory. 1994. Exploration in Automatic Thesaurus Discovery. Kluwer Academic Publishers. find
  7. Harris, Zelig S. Mathematical structures of language. Wiley, 1968. find
  8. Hearst, Marti. 1992. Automatic acquisition of hyponyms from large text corpora. In Proc. of COLING-92. Nantes, France. find
  9. Keller, Frank, Maria Lapata, and Olga Ourioupina. 2002. Using the Web to Overcome Data Sparseness. In Jan Hajic and Yuji Matsumoto, eds., In Proc. of EMNLP-02. Philadelphia, PA. find
  10. Lee, Lillian. 1997. Similarity-Based Approaches to Natural Language Processing. Ph.D. thesis, Harvard University, Cambridge, MA. find
  11. Lin, Dekang. 1993. Principle-Based Parsing without Overgeneration. In Proc. of ACL-93. Columbus, Ohio. . find
  12. Lin, Dekang. 1998. Automatic Retrieval and Clustering of Similar Words. In Proc. of COLING–ACL98, Montreal, Canada. find our card
  13. Moldovan, Dan, Badulescu, A., Tatu, M., Antohe, D., and Girju, R. 2004. Models for the semantic classification of noun phrases. In Proc. of HLT/NAACL-2004 Workshop on Computational Lexical Semantics. Boston. find
  14. Pado, Sebastian and Mirella Lapata. 2003. Constructing semantic space models from parsed corpora. In Proc. of ACL-03, Sapporo, Japan. find
  15. Pantel, Patrick and Dekang Lin. 2002. Discovering Word Senses from Text. In Proc. of ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD-02). Edmonton, Canada. find
  16. Pereira, Fernando, Tishby Naftali, and Lee Lillian. 1993. Distributional clustering of English words. In Proc. of ACL-93. Columbus, Ohio. find our card
  17. Ravichandran, Deepak and Eduard Hovy. 2002. Learning Surface Text Patterns for a Question Answering System. In Proc. of ACL-02. Philadelphia, PA. find our card
  18. Ruge, Gerda. 1992. Experiments on linguistically-based term associations. Information Processing & Management, 28(3), pp. 317–332. find
  19. Weeds, Julie and David Weir. 2003. A General Framework for Distributional Similarity. In Proc. of EMNLP-03. Sapporo, Japan. find
  20. Weeds, Julie, D. Weir, D. McCarthy. 2004. Characterizing Measures of Lexical Distributional Similarity. In Proc. of Coling-04. Geneva, Switzerland. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/geffet-2005-distributional-inclusion-hypotheses-lexical-entailment/