Elamur Research⌕
☰
home / papers / 2009 / Bootstrapping Distributional Feature Vector Quality

Bootstrapping Distributional Feature Vector Quality

Paper · 2009 · Maayan Zhitomirsky-Geffet and Ido Dagan

Summary

Distributional similarity methods often group words that are only loosely related. For example, distant co-hyponyms such as party and company come out close to country. This makes such methods a weak source of substitutable lexical entailment for QA, IR and textual entailment systems. The authors trace much of this looseness to poorly weighted feature vectors: common weightings such as PMI fail to rank the most meaning-bearing contexts highest. Their fix is a bootstrapping reweighting step. A conventional measure first builds a rough similarity space, and then features a word shares with its nearest neighbours are boosted, since shared contexts are assumed to be the most characteristic. Features are syntactic dependency triples produced by the Minipar parser. Evaluation uses two settings: human lexical-entailment judgments and a pseudo-word sense disambiguation task. A new diagnostic metric supports the analysis. Bootstrapping helps across several similarity measures and weighting functions and allows strong feature pruning. It therefore offers a measure-independent way to obtain cleaner distributional vectors.

Key points

  • Features are triples of context word, Minipar dependency relation, and head/modifier role. Bootstrapping was applied over three similarity measures (LIN, WJ, COS) and two weighting functions, with consistent gains.
  • The gold standard for the refined substitutable lexical entailment relation was judged by three assessors, with high inter-annotator agreement.
  • In pseudo-WSD, error rates fell from 0.157 to 0.133 (LIN), 0.150 to 0.132 (WJ) and 0.155 to 0.145 (COS). The LIN reduction exceeds 15%, compared with about 6% reported by Weeds and Weir (2005).
  • The authors introduce the average common-feature rank ratio (acfr-ratio). They present it as the first metric that judges feature weighting directly, without plugging it into a similarity measure.
  • Characteristic features concentrate at the top of bootstrapped vectors, enabling massive feature reduction. The method also underlies the feature inclusion scheme of Geffet and Dagan (2005).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Bar-Ilan UniversityRamat Gan · Israel141 works · 23,225 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 works

Cite

Cite as Maayan Zhitomirsky-Geffet and Ido Dagan. 2009. Bootstrapping Distributional Feature Vector Quality. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/coli.08-032-r1-06-96

@article{zhitomirskygeffet2009bootstrapping,
  title = {{Bootstrapping Distributional Feature Vector Quality}},
  author = {Maayan Zhitomirsky-Geffet and Ido Dagan},
  journal = {Computational Linguistics},
  year = {2009},
  doi = {10.1162/coli.08-032-r1-06-96},
  url = {https://doi.org/10.1162/coli.08-032-r1-06-96}
}

References

54 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Adams, Rod. 2006. Textual entailment through extended lexical overlap. In Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment, pages 68–73, Venice. find
  2. Bar-Haim, Roy, Ido Dagan, Bill Dolan, Lisa Ferro, Danilo Giampiccolo, Bernardo Magnini, and Idan Szpektor. 2006. The second PASCAL recognising textual entailment challenge. In Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment, pages 1–9, Venice. find
  3. Baroni, Marco and S. Vegnaduzzo. 2004. Identifying subjective adjectives through web-based mutual information. In Proceedings of KONVENS–04, pages 17–24, Vienna. find
  4. Barzilay, Regina and Kathleen McKeown. 2001. Extracting paraphrases from a parallel corpus. In Proceedings of ACL / EACL–01, pages 50–57, Toulouse.
  5. Caraballo, Sharon A. 1999. Automatic construction of a hypernym-labeled noun hierarchy from text. In Proceedings of ACL–99, pages 120–126, College Park, MD.
  6. Chklovski, Timothy and Patrick Pantel. 2004. VerbOcean: Mining the web for fine-grained Semantic Verb Relations. In Proceedings of EMNLP–04, pages 33–40, Barcelona. find
  7. Church, Kenneth W. and Hanks Patrick. 1990. Word association norms, mutual information, and lexicography. Computational Linguistics, 16(1):22–29.
  8. Curran, James R. 2004. From Distributional to Semantic Similarity. Ph.D. Thesis, School of Informatics of the University of Edinburgh, Scotland. find
  9. Curran, James R. 2005. Supersense tagging of unknown nouns using semantic similarity. In Proceedings of ACL–2005, pages 26–33, Ann Arbor, MI.
  10. Curran, James R. and Marc Moens. 2002. Improvements in automatic thesaurus extraction. In Proceedings of the Workshop on Unsupervised Lexical Acquisition, pages 59–67, Philadelphia, PA.
  11. Dagan, Ido. 2000. Contextual Word Similarity. In Rob Dale, Hermann Moisl, and Harold Somers, editors, Handbook of Natural Language Processing. Marcel Dekker Inc, Chapter 19, pages 459–476, New York, NY. find
  12. Dagan, Ido, Oren Glickman, and Bernardo Magnini. 2006. The PASCAL recognising textual entailment challenge. Lecture Notes in Computer Science, 3944:177–190. find
  13. Dagan, Ido, Lillian Lee, and Fernando Pereira. 1999. Similarity-based models of co-occurrence probabilities. Machine Learning, 34(1-3):43–69. find
  14. Dagan, Ido, Shaul Marcus, and Shaul Markovitch. 1995. Contextual word similarity and estimation from sparse data. Computer, Speech and Language, 9:123–152. find
  15. Dunning, Ted E. 1993. Accurate methods for the statistics of surprise and coincidence. Computational Linguistics, 19(1):61–74.
  16. Essen, U., and V. Steinbiss. 1992. Co-occurrence smoothing for stochastic language modeling. In ICASSP–92, 1:161–164, Piscataway, NJ. find
  17. Fellbaum, Christiane, editor. 1998. WordNet: An Electronic Lexical Database. MIT Press, Cambridge, MA. find
  18. Ferrandez, O., R. M. Terol, R. Munoz, P. Martinez-Barco, and M. Palomar. 2006. An approach based on logic forms and WordNet relationships to textual entailment performance. In Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment, pages 22–26, Venice. find
  19. Gasperin, Caroline and Renata Vieira. 2004. Using word similarity lists for resolving indirect anaphora. In Proceedings of ACL–04 Workshop on Reference Resolution, pages 40–46, Barcelona.
  20. Gauch, Susan, J. Wang, and S. Mahesh Rachakonda. 1999. A corpus analysis approach for automatic query expansion and its extension to multiple databases. ACM Transactions on Information Systems (TOIS), 17(3):250–269. find
  21. Geffet, Maayan. 2006. Refining the Distributional Similarity Scheme for Lexical Entailment. Ph.D. Thesis. School of Computer Science and Engineering, Hebrew University, Jerusalem, Israel. find
  22. Geffet, Maayan and Ido Dagan. 2004. Feature vector quality and distributional similarity. In Proceedings of COLING–04, Article number: 247, Geneva. find
  23. Geffet, Maayan and Ido Dagan. 2005. The distributional inclusion hypotheses and lexical entailment. In Proceedings of ACL–05, pages 107–114, Ann Arbor, MI.
  24. Gliozzo, Alfio, Claudio Giuliano, and Carlo Strapparava. 2005. Domain kernels for word sense disambiguation. In Proceedings of ACL–05, pages 403–410, Ann Arbor, MI. find
  25. Grefenstette, Gregory. 1994. Exploration in Automatic Thesaurus Discovery. Kluwer Academic Publishers, Norwell, MA. find
  26. Harris, Zelig S. 1968. Mathematical structures of language. Wiley, New Jersey. find
  27. Hindle, D. 1990. Noun classification from predicate-argument structures. In Proceedings of ACL–90, pages 268–275, Pittsburgh, PA.
  28. Jijkoun, Valentin and Maarten de Rijke. 2005. Recognizing textual entailment: Is word similarity enough? In Joaquin Quinonero Candela, Ido Dagan, Bernardo Magnini, and Florence d’Alche-Buc, editors, Machine Learning Challenges, Evaluating Predictive Uncertainty, Visual Object Classification and Reco… find
  29. Karov, Y. and S. Edelman. 1996. Learning similarity-based word sense disambiguation from sparse data. In E. Ejerhed and I. Dagan, editors, Fourth Workshop on Very Large Corpora. Association for Computational Linguistics, Somerset, NJ, pages 42–55. find
  30. Landis, J. R. and G. G. Koch. 1997. The measurements of observer agreement for categorical data. Biometrics, 33:159–174. find
  31. Lee, Lillian. 1997. Similarity-Based Approaches to Natural Language Processing. Ph.D. thesis, Harvard University, Cambridge, MA. find
  32. Lee, Lillian.1999. Measures of distributional similarity. In Proceedings of ACL–99, pages 25–32, College Park, MD.
  33. Lin, Dekang. 1993. Principle-based parsing without overgeneration. In Proceedings of ACL–93, pages 112–120, Columbus, OH. find
  34. Lin, Dekang. 1998. Automatic retrieval and clustering of similar words. In Proceedings of COLING/ACL–98, pages 768–774, Montreal.
  35. Lin, Dekang and Patrick Pantel. 2001. Discovery of inference rules for question answering. Natural Language Engineering, 7(4):343–360. find
  36. Luk, Alpha K. 1995. Statistical sense disambiguation with relatively small corpora using dictionary definitions. In Proceedings of ACL–95, pages 181–188, Cambridge, MA. find
  37. Manning, Christopher D. and Hinrich Schutze. 1999. Foundations of Statistical Natural Language Processing. MIT Press, Cambridge, MA. find
  38. Mirkin, Shachar, Ido Dagan, and Maayan Geffet. 2006. Integrating pattern-based and distributional similarity methods for lexical entailment acquisition. In Proceedings of the COLING/ACL–06 Main Conference Poster Sessions, pages 579–586, Sydney. find
  39. Ng, H. T. 1997. Exemplar-based word sense disambiguation: Some recent improvements. In Proceedings of EMNLP– 97, pages 208–213, Providence, RI. find
  40. Ng, H. T. and H. B. Lee. 1996. Integrating multiple knowledge sources to disambiguate word sense: An exemplar-based approach. In Proceedings of ACL–1996, pages 40–47, Santa Cruz, CA.
  41. Nicholson, Jeremy, Nicola Stokes, and Timothy Baldwin. 2006. Detecting entailment using an extended implementation of the basic elements overlap metric. In Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment, pages 122–127, Venice. find
  42. Pado, Sebastian and Mirella Lapata. 2007. Dependency-based construction of semantic space models. Computational Linguistics, 33(2):161–199.
  43. Pantel, Patrick and Deepak Ravichandran. 2004. Automatically labeling semantic classes. In Proceedings of HLT/NAACL–04, pages 321–328, Boston, MA.
  44. Pantel, Patrick, D. Ravichandran, and E. Hovy. 2004. Towards terascale knowledge acquisition. In Proceedings of COLING–04, Article number: 771, Geneva. find
  45. Pekar, Viktor, M. Krkoska, and S. Staab. 2004. Feature weighting for co-occurrence-based classification of Words. In Proceedings of COLING–04, Article number: 799, Geneva. find
  46. Pereira, Fernando, Naftali Tishby, and Lillian Lee. 1993. Distributional clustering of English words. In Proceedings of ACL–93, pages 183–190, Colombus, OH.
  47. Ruge, Gerda. 1992. Experiments on linguistically-based term associations. Information Processing & Management, 28(3):317–332. find
  48. Salton, G. and M. J. McGill. 1983. Introduction to Modern Information Retrieval. McGraw-Hill, New York, NY. find
  49. Szpektor, Idan, H. Tanev, Ido Dagan, and B. Coppola. 2004. Scaling web-based acquisition of entailment relations. In Proceedings of EMNLP–04, pages 41–48, Barcelona.
  50. Vanderwende, Lucy, Arul Menezes, and Rion Snow. 2006. Microsoft research at RTE-2: Syntactic contributions in the entailment task: An implementation. In Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment, pages 27–32, Venice. find
  51. Weeds, Julie and David Weir. 2005. Co-occurrence retrieval: A flexible framework for lexical distributional similarity. Computational Linguistics, 31(4):439–476.
  52. Weeds, Julie, D. Weir, and D. McCarthy. 2004. Characterizing measures of lexical distributional similarity. In Proceedings of COLING–04, pages 1015–1021, Switzerland. find
  53. Widdows, D. 2003. Unsupervised methods for developing taxonomies by combining syntactic and statistical information. In Proceedings of HLT/NAACL 2003, pages 197–204, Edmonton. find
  54. Yarowsky, D. 1992. Word-sense disambiguation using statistical models of Roget’s categories trained on large corpora. In Proceedings of COLING–92, pages 454–460, Nantes.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/geffet-2009-bootstrapping-distributional-feature-vector-quality/