Elamur Research⌕
☰
home / papers / 2010 / From Frequency to Meaning: Vector Space Models of Semantics

From Frequency to Meaning: Vector Space Models of Semantics

Summary

This survey is devoted to vector space models of semantics (VSM). In this approach, a word, phrase or document is represented as a point in a high-dimensional space, and points that are close together are taken to be close in meaning. The authors organize a large body of disparate work according to a single principle: the structure of the matrix. They identify three types of matrices: term–document, word–context and pair–pattern, and each type opens up its own class of applications. According to the authors' definition, an essential feature of a VSM is that the values of the matrix elements must derive from event frequencies in a corpus. This is precisely what links such models to the distributional hypothesis. For each class, one open-source software project is examined, and the linguistic and mathematical processing of matrices is described separately. Particular attention is given to pair–pattern matrices: they take word order into account and make it possible to compare semantic relations rather than just individual words. The work is important because it provides a coherent framework for the entire field and outlines open problems: compositionality of meaning, word order and the limits of the distributional approach. The authors argue that VSMs were at the time arguably the most successful approach to computational semantics.

Key points

  • The three classes of VSM are defined by the structure of the matrix: term–document (retrieval and document similarity), word–context (word similarity, e.g. LSA and HAL) and pair–pattern (similarity of relations between word pairs). The authors consider the choice of matrix more important than the specific linguistic or mathematical processing.
  • Reported results: the Rapp (2003) model correctly answered 92.5% of TOEFL synonym test questions, whereas humans score 64.5% on average. The Turney (2006) model achieved 56% on SAT analogy questions versus an average of 57% for humans.
  • The origins of the approach lie in the SMART information retrieval system (Salton, 1971; Salton et al., 1975). VSMs are learned automatically from a corpus, such as the BNC. Alternative methods rely on hand-built lexicons such as WordNet or Roget's Thesaurus, and compiling a corpus for a new language is much easier than building a lexicon.
  • The critics' main objection is that term–document and word–context models ignore word order, so "house boat" and "boat house" receive the same vector. According to Landauer (2002), word order accounts for about 20% of the meaning of an English text. Responses to this problem include the compositional models of Mitchell & Lapata (2008), the tensor product in Clark & Pulman (2007), and pair–pattern matrices.
  • Open questions: how to represent arbitrary statements of first-order logic, and whether statistics of word usage are sufficient to capture all of meaning. The authors anticipate the emergence of new types of matrices and higher-order tensors.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1613/jair.2934 · arXiv 1003.1141

@misc{turney2010frequency,
  title = {{From Frequency to Meaning: Vector Space Models of Semantics}},
  author = {Turney, Peter D. and Pantel, Patrick},
  year = {2010},
  doi = {10.1613/jair.2934},
  eprint = {1003.1141},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1613/jair.2934}
}

References

188 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Acar, E., & Yener, B. (2009). Unsupervised multiway data analysis: A literature survey. IEEE Transactions on Knowledge and Data Engineering, 21 (1), 6–20. find
  2. Agirre, E., & Edmonds, P. G. (2006). Word Sense Disambiguation: Algorithms and Applications. Springer. find
  3. Ando, R. K. (2000). Latent semantic space: Iterative scaling improves precision of inter-document similarity measurement. In Proceedings of the 23rd Annual ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR-2000), pp. 216–223. find
  4. Ando, R. K., & Zhang, T. (2005). A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research, 6, 1817– 1853. find
  5. Baeza-Yates, R., & Ribeiro-Neto, B. (1999). Modern Information Retrieval. Addison Wesley. find
  6. Barr, C., Jones, R., & Regelson, M. (2008). The linguistic structure of English web-search queries. In Conference on Empirical Methods in Natural Language Processing (EMNLP). find
  7. Bayardo, R. J., Ma, Y., & Srikant, R. (2007). Scaling up all pairs similarity search. In Proceedings of the 16th international conference on World Wide Web (WWW ’07), pp. 131–140, New York, NY. ACM. find
  8. Biçici, E., & Yuret, D. (2006). Clustering word pairs to answer analogy questions. In Proceedings of the Fifteenth Turkish Symposium on Artificial Intelligence and Neural Networks (TAINN 2006), Akyaka, Mugla, Turkey. find
  9. Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet Allocation. Journal of Machine Learning Research, 3, 993–1022. find our card
  10. Brand, M. (2006). Fast low-rank modifications of the thin singular value decomposition. Linear Algebra and Its Applications, 415 (1), 20–30. find
  11. Breese, J., Heckerman, D., & Kadie, C. (1998). Empirical analysis of predictive algorithms for collaborative filtering. In Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence, pp. 43–52. Morgan Kaufmann. find
  12. Brin, S., & Page, L. (1998). The anatomy of a large-scale hypertextual Web search engine. In Proceedings of the Seventh World Wide Web Conference (WWW7), pp. 107–117. find our card
  13. Broder, A. (1997). On the resemblance and containment of documents. In In Compression and Complexity of Sequences (SEQUENCES’97), pp. 21–29. IEEE Computer Society. find
  14. Budanitsky, A., & Hirst, G. (2001). Semantic distance in WordNet: An experimental, application-oriented evaluation of five measures. In Proceedings of the Workshop on WordNet and Other Lexical Resources, Second Meeting of the North American Chapter of the Association for Computational Linguistics (N… find
  15. Budanitsky, A., & Hirst, G. (2006). Evaluating wordnet-based measures of semantic distance. Computational Linguistics, 32 (1), 13–47. find
  16. Bullinaria, J., & Levy, J. (2007). Extracting semantic representations from word co-occurrence statistics: A computational study. Behavior Research Methods, 39 (3), 510–526. find
  17. Buntine, W., & Jakulin, A. (2006). Discrete component analysis. In Subspace, Latent Structure and Feature Selection: Statistical and Optimization Perspectives Workshop at SLSFS 2005, pp. 1–33, Bohinj, Slovenia. Springer. find
  18. Cafarella, M. J., Banko, M., & Etzioni, O. (2006). Relational web search. Tech. rep., University of Washington, Department of Computer Science and Engineering. Technical Report 2006-04-02. find
  19. Cao, G., Nie, J.-Y., & Bai, J. (2005). Integrating word relationships into language models. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’05), pp. 298–305, New York, NY. ACM. find
  20. Cao, H., Jiang, D., Pei, J., He, Q., Liao, Z., Chen, E., & Li, H. (2008). Context-aware query suggestion by mining click-through and session data. In Proceeding of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD ’08), pp. 875–883. ACM. find
  21. Carroll, J. D., & Chang, J.-J. (1970). Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition. Psychometrika, 35 (3), 283–319. find
  22. Chang, W., Pantel, P., Popescu, A.-M., & Gabrilovich, E. (2009). Towards intent-driven bidterm suggestion. In Proceedings of WWW-09 (Short Paper), Madrid, Spain. find
  23. Charikar, M. S. (2002). Similarity estimation techniques from rounding algorithms. In Proceedings of the thiry-fourth annual ACM symposium on Theory of computing (STOC ’02), pp. 380–388. ACM. find
  24. Chew, P., Bader, B., Kolda, T., & Abdelali, A. (2007). Cross-language information retrieval using PARAFAC2. In Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD07), pp. 143–152. ACM Press. find
  25. Chiarello, C., Burgess, C., Richards, L., & Pollock, A. (1990). Semantic and associative priming in the cerebral hemispheres: Some words do, some words don’t . . . sometimes, some places. Brain and Language, 38, 75–104. find
  26. Chklovski, T., & Pantel, P. (2004). VerbOcean: Mining the web for fine-grained semantic verb relations. In Proceedings of Experimental Methods in Natural Language Processing 2004 (EMNLP-04), pp. 33–40, Barcelona, Spain. find
  27. Choi, F. Y. Y. (2000). Advances in domain independent linear text segmentation. In Proceedings of the 1st Meeting of the North American Chapter of the Association for Computational Linguistics, pp. 26–33. find our card
  28. Chu-carroll, J., & Carpenter, B. (1999). Vector-based natural language call routing. Computational Linguistics, 25 (3), 361–388. find
  29. Church, K. (1995). One term or two?. In Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 310–318. find
  30. Church, K., & Hanks, P. (1989). Word association norms, mutual information, and lexicography. In Proceedings of the 27th Annual Conference of the Association of Computational Linguistics, pp. 76–83, Vancouver, British Columbia. find our card
  31. Clark, S., & Pulman, S. (2007). Combining symbolic and distributional models of meaning. In Proceedings of AAAI Spring Symposium on Quantum Interaction, pp. 52–55. find
  32. Collobert, R., & Weston, J. (2008). A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th International Conference on Machine Learning (ICML-08), pp. 160–167. find
  33. Croft, W. B. (1977). Clustering large files of documents using the single-link method. Journal of the American Society for Information Science, 28 (6), 341–344. find
  34. Crouch, C. J. (1988). A cluster-based approach to thesaurus construction. In Proceedings of the 11th Annual International ACM SIGIR Conference, pp. 309–320, Grenoble, France. find
  35. Curran, J. R., & Moens, M. (2002). Improvements in automatic thesaurus extraction. In Unsupervised Lexical Acquisition: Proceedings of the Workshop of the ACL Special Interest Group on the Lexicon (SIGLEX), pp. 59–66, Philadelphia, PA. find
  36. Cutting, D. R., Karger, D. R., Pedersen, J. O., & Tukey, J. W. (1992). Scatter/gather: a cluster-based approach to browsing large document collections. In Proceedings of the 15th Annual International ACM SIGIR Conference, pp. 318–329. find
  37. Dagan, I., Lee, L., & Pereira, F. C. N. (1999). Similarity-based models of word cooccurrence probabilities. Machine Learning, 34 (1–3), 43–69. find
  38. Dang, H. T., Lin, J., & Kelly, D. (2006). Overview of the TREC 2006 question answering track. In Proceedings of the Fifteenth Text REtrieval Conference (TREC 2006). find
  39. Dasarathy, B. (1991). Nearest Neighbor (NN) Norms: NN Pattern Classification Techniques. IEEE Computer Society Press. find
  40. Davidov, D., & Rappoport, A. (2008). Unsupervised discovery of generic relationships using pattern clusters and its evaluation by automatically generated SAT analogy questions. In Proceedings of the 46th Annual Meeting of the ACL and HLT (ACL-HLT-08), pp. 692–700, Columbus, Ohio. find
  41. Dean, J., & Ghemawat, S. (2008). MapReduce: Simplified data processing on large clusters. Communications of the ACM, 51 (1), 107–113. find
  42. Deerwester, S. C., Dumais, S. T., Landauer, T. K., Furnas, G. W., & Harshman, R. A. (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science (JASIS), 41 (6), 391–407. find
  43. Elsayed, T., Lin, J., & Oard, D. (2008). Pairwise document similarity in large collections with mapreduce. In Proceedings of Association for Computational Linguistics and Human Language Technology Conference 2008 (ACL-08: HLT), Short Papers, pp. 265–268, Columbus, Ohio. Association for Computational… find
  44. Erk, K. (2007). A simple, similarity-based model for selectional preferences. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pp. 216– 223,, Prague, Czech Republic. find
  45. Erk, K., & Padó, S. (2008). A structured vector space model for word meaning in context. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing (EMNLP-08), pp. 897–906, Honolulu, HI. find our card
  46. Fellbaum, C. (Ed.). (1998). WordNet: An Electronic Lexical Database. MIT Press. find
  47. Firth, J. R. (1957). A synopsis of linguistic theory 1930–1955. In Studies in Linguistic Analysis, pp. 1–32. Blackwell, Oxford. find
  48. Foltz, P. W., Laham, D., & Landauer, T. K. (1999). The intelligent essay assessor: Applications to educational technology. Interactive Multimedia Electronic Journal of Computer-Enhanced Learning, 1 (2). find
  49. Forman, G. (2003). An extensive empirical study of feature selection metrics for text classification. Journal of Machine Learning Research, 3, 1289–1305. find
  50. French, R. M., & Labiouse, C. (2002). Four problems with extracting human semantics from large text corpora. In Proceedings of the 24th Annual Conference of the Cognitive Science Society. find
  51. Furnas, G. W., Landauer, T. K., Gomez, L. M., & Dumais, S. T. (1983). Statistical semantics: Analysis of the potential performance of keyword information systems. Bell System Technical Journal, 62 (6), 1753–1806. find
  52. Gentner, D. (1983). Structure-mapping: A theoretical framework for analogy. Cognitive Science, 7 (2), 155–170. find
  53. Gilbert, J. R., Moler, C., & Schreiber, R. (1992). Sparse matrices in MATLAB: Design and implementation. SIAM Journal on Matrix Analysis and Applications, 13 (1), 333–356. find
  54. Girju, R., Nakov, P., Nastase, V., Szpakowicz, S., Turney, P., & Yuret, D. (2007). Semeval-2007 task 04: Classification of semantic relations between nominals. In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval 2007), pp. 13–18, Prague, Czech Republic. find
  55. Gleich, D., & Zhukov, L. (2004). SVD based term suggestion and ranking system. In Proceedings of the Fourth IEEE International Conference on Data Mining (ICDM ’04), pp. 391–394. IEEE Computer Society. find
  56. Golub, G. H., & Van Loan, C. F. (1996). Matrix Computations (Third edition). Johns Hopkins University Press, Baltimore, MD. find
  57. Gorman, J., & Curran, J. R. (2006). Scaling distributional similarity to large corpora. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics (ACL 2006), pp. 361–368. Association for Computational… find
  58. Gorrell, G. (2006). Generalized Hebbian algorithm for incremental singular value decomposition in natural language processing. In Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics (EACL-06), pp. 97–104. find
  59. Gospodnetic, O., & Hatcher, E. (2004). Lucene in Action. Manning Publications. find
  60. Grefenstette, G. (1994). Explorations in Automatic Thesaurus Discovery. Kluwer. find
  61. Harris, Z. (1954). Distributional structure. Word, 10 (23), 146–162. find
  62. Harshman, R. (1970). Foundations of the parafac procedure: Models and conditions for an “explanatory” multi-modal factor analysis. UCLA Working Papers in Phonetics, 16. find
  63. Hearst, M. (1997). Texttiling: Segmenting text into multi-paragraph subtopic passages. Computational Linguistics, 23 (1), 33–64. find
  64. Hirst, G., & St-Onge, D. (1998). Lexical chains as representations of context for the detection and correction of malapropisms. In Fellbaum, C. (Ed.), WordNet: An Electronic Lexical Database, pp. 305–332. MIT Press. find
  65. Hofmann, T. (1999). Probabilistic Latent Semantic Indexing. In Proceedings of the 22nd Annual ACM Conference on Research and Development in Information Retrieval (SI-GIR ’99), pp. 50–57, Berkeley, California. find
  66. Huang, C.-K., Chien, L.-F., & Oyang, Y.-J. (2003). Relevant term suggestion in interactive web search based on contextual information in query session logs. Journal of the American Society for Information Science and Technology, 54 (7), 638–649. find
  67. Hull, D. (1996). Stemming algorithms: A case study for detailed evaluation. Journal of the American Society for Information Science, 47 (1), 70–84. find
  68. Jain, A., Murty, N., & Flynn, P. (1999). Data clustering: A review. ACM Computing Surveys, 31 (3), 264–323. find
  69. Jarmasz, M., & Szpakowicz, S. (2003). Roget’s thesaurus and semantic similarity. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP-03), pp. 212–219, Borovets, Bulgaria. find
  70. Jiang, J. J., & Conrath, D. W. (1997). Semantic similarity based on corpus statistics and lexical taxonomy. In Proceedings of the International Conference on Research in Computational Linguistics (ROCLING X), pp. 19–33, Tapei, Taiwan. find
  71. Johnson, H., & Martin, J. (2003). Unsupervised learning of morphology for English and Inuktitut. In Proceedings of HLT-NAACL 2003, pp. 43–45. find
  72. Jones, M. P., & Martin, J. H. (1997). Contextual spelling correction using latent semantic analysis. In Proceedings of the Fifth Conference on Applied Natural Language Processing, pp. 166–173, Washington, DC. find
  73. Jones, R., Rey, B., Madani, O., & Greiner, W. (2006). Generating query substitutions. In Proceedings of the 15th international conference on World Wide Web (WWW ’06), pp. 387–396, New York, NY. ACM. find
  74. Jones, W. P., & Furnas, G. W. (1987). Pictures of relevance: A geometric analysis of similarity measures. Journal of the American Society for Information Science, 38 (6), 420–442. find
  75. Kanerva, P. (1993). Sparse distributed memory and related models. In Hassoun, M. H. (Ed.), Associative neural memories, pp. 50–76. Oxford University Press, New York, NY. find
  76. Karlgren, J., & Sahlgren, M. (2001). From words to understanding. In Uesaka, Y., Kanerva, P., & Asoh, H. (Eds.), Foundations of Real-World Intelligence, pp. 294–308. CSLI Publications. find
  77. Kim, S.-M., Pantel, P., Chklovski, T., & Pennacchiotti, M. (2006). Automatically assessing review helpfulness. In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pp. 423–430. find
  78. Kolda, T., & Bader, B. (2009). Tensor decompositions and applications. SIAM Review, 51 (3), 455–500. find
  79. Konchady, M. (2008). Building Search Applications: Lucene, LingPipe, and Gate. Mustru Publishing. find
  80. Kraaij, W., & Pohlmann, R. (1996). Viewing stemming as recall enhancement. In Proceedings of the 19th Annual International ACM SIGIR Conference, pp. 40–48. find
  81. Lakoff, G. (1987). Women, Fire, and Dangerous Things. University Of Chicago Press, Chicago, IL. find
  82. Landauer, T. K. (2002). On the computational basis of learning and cognition: Arguments from LSA. In Ross, B. H. (Ed.), The Psychology of Learning and Motivation: Advances in Research and Theory, Vol. 41, pp. 43–84. Academic Press. find
  83. Landauer, T. K., & Dumais, S. T. (1997). A solution to Plato’s problem: The latent semantic analysis theory of the acquisition, induction, and representation of knowledge. Psychological Review, 104 (2), 211–240. find
  84. Landauer, T. K., & Littman, M. L. (1990). Fully automatic cross-language document retrieval using latent semantic indexing. In Proceedings of the Sixth Annual Conference of the UW Centre for the New Oxford English Dictionary and Text Research, pp. 31– 38, Waterloo, Ontario. find
  85. Landauer, T. K., McNamara, D. S., Dennis, S., & Kintsch, W. (2007). Handbook of Latent Semantic Analysis. Lawrence Erlbaum, Mahwah, NJ. find
  86. Lavrenko, V., & Croft, W. B. (2001). Relevance based language models. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’01), pp. 120–127, New York, NY. ACM. find
  87. Leacock, C., & Chodrow, M. (1998). Combining local context and WordNet similarity for word sense identification. In Fellbaum, C. (Ed.), WordNet: An Electronic Lexical Database. MIT Press. find
  88. Leacock, C., Towell, G., & Voorhees, E. (1993). Corpus-based statistical sense resolution. In Proceedings of the ARPA Workshop on Human Language Technology, pp. 260–265. find
  89. Lee, D. D., & Seung, H. S. (1999). Learning the parts of objects by nonnegative matrix factorization. Nature, 401, 788–791. find
  90. Lee, L. (1999). Measures of distributional similarity. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics, pp. 25–32. find
  91. Lemaire, B., & Denhière, G. (2006). Effects of high-order co-occurrences on word semantic similarity. Current Psychology Letters: Behaviour, Brain & Cognition, 18 (1). find
  92. Lin, D. (1998). Automatic retrieval and clustering of similar words. In roceedings of the 17th international conference on Computational linguistics, pp. 768–774. Association for Computational Linguistics. find our card
  93. Lin, D., & Pantel, P. (2001). DIRT – discovery of inference rules from text. In Proceedings of ACM SIGKDD Conference on Knowledge Discovery and Data Mining 2001, pp. 323–328. find
  94. Linden, G., Smith, B., & York, J. (2003). Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Internet Computing, 76–80. find
  95. Liu, X., & Croft, W. B. (2005). Statistical language modeling for information retrieval. Annual Review of Information Science and Technology, 39, 3–28. find
  96. Lovins, J. B. (1968). Development of a stemming algorithm. Mechanical Translation and Computational Linguistics, 11, 22–31. find
  97. Lowe, W. (2001). Towards a theory of semantic space. In Proceedings of the Twenty-first Annual Conference of the Cognitive Science Society, pp. 576–581. find
  98. Lund, K., & Burgess, C. (1996). Producing high-dimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, Instruments, and Computers, 28 (2), 203– 208. find
  99. Lund, K., Burgess, C., & Atchley, R. A. (1995). Semantic and associative priming in high-dimensional semantic space. In Proceedings of the 17th Annual Conference of the Cognitive Science Society, pp. 660–665. find
  100. Manning, C., & Schütze, H. (1999). Foundations of Statistical Natural Language Processing. MIT Press, Cambridge, MA. find
  101. Manning, C. D., Raghavan, P., & Schütze, H. (2008). Introduction to Information Retrieval. Cambridge University Press, Cambridge, UK. find
  102. Miller, G., Leacock, C., Tengi, R., & Bunker, R. (1993). A semantic concordance. In Proceedings of the 3rd DARPA Workshop on Human Language Technology, pp. 303– 308. find our card
  103. Minnen, G., Carroll, J., & Pearce, D. (2001). Applied morphological processing of English. Natural Language Engineering, 7 (3), 207–223. find
  104. Mitchell, J., & Lapata, M. (2008). Vector-based models of semantic composition. In Proceedings of ACL-08: HLT, pp. 236–244, Columbus, Ohio. Association for Computational Linguistics. find
  105. Mitchell, T. (1997). Machine Learning. McGraw-Hill, Columbus, OH. find
  106. Mohammad, S., & Hirst, G. (2006). Distributional measures of concept-distance: A task-oriented evaluation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP-2006), pp. 35–43. find
  107. Monay, F., & Gatica-Perez, D. (2003). On image auto-annotation with latent space models. In Proceedings of the Eleventh ACM International Conference on Multimedia, pp. 275–278. find
  108. Morris, J., & Hirst, G. (2004). Non-classical lexical semantic relations. In Workshop on Computational Lexical Semantics, HLT-NAACL-04, Boston, MA. find
  109. Nakov, P., & Hearst, M. (2007). UCB: System description for SemEval Task 4. In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval 2007), pp. 366–369, Prague, Czech Republic. find
  110. Nakov, P., & Hearst, M. (2008). Solving relational similarity problems using theweb as a corpus. In Proceedings of ACL-08: HLT, pp. 452–460, Columbus, Ohio. find
  111. Nastase, V., Sayyad-Shirabad, J., Sokolova, M., & Szpakowicz, S. (2006). Learning noun-modifier semantic relations with corpus-based and WordNet-based features. In Proceedings of the 21st National Conference on Artificial Intelligence (AAAI-06), pp. 781–786. find
  112. Nastase, V., & Szpakowicz, S. (2003). Exploring noun-modifier semantic relations. In Fifth International Workshop on Computational Semantics (IWCS-5), pp. 285–301, Tilburg, The Netherlands. find
  113. Niwa, Y., & Nitta, Y. (1994). Co-occurrence vectors from corpora vs. distance vectors from dictionaries. In Proceedings of the 15th International Conference On Computational Linguistics, pp. 304–309, Kyoto, Japan. find
  114. Nosofsky, R. (1986). Attention, similarity, and the identification-categorization relationship. Journal of Experimental Psychology: General, 115 (1), 39–57. find
  115. Ogden, C. K. (1930). Basic English: A General Introduction with Rules and Grammar. Kegan Paul, Trench, Trubner and Co. find
  116. Padó, S., & Lapata, M. (2003). Constructing semantic space models from parsed corpora. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, pp. 128–135, Sapporo, Japan. find
  117. Padó, S., & Lapata, M. (2007). Dependency-based construction of semantic space models. Computational Linguistics, 33 (2), 161–199. find our card
  118. Pang, B., Lee, L., & Vaithyanathan, S. (2002). Thumbs up? sentiment classification using machine learning techniques. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 79–86, Philadelphia, PA. find our card
  119. Pantel, P. (2005). Inducing ontological co-occurrence vectors. In Proceedings of Association for Computational Linguistics (ACL-05), pp. 125–132. find
  120. Pantel, P., & Lin, D. (1998). Spamcop: A spam classification and organization program. In Learning for Text Categorization: Papers from the AAAI 1998 Workshop, pp. 95–98. find
  121. Pantel, P., & Lin, D. (2002a). Discovering word senses from text. In Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 613–619, Edmonton, Canada. find
  122. Pantel, P., & Lin, D. (2002b). Document clustering with committees. In Proceedings of the 25th Annual International ACM SIGIR Conference, pp. 199–206. find
  123. Paşca, M., Lin, D., Bigham, J., Lifchits, A., & Jain, A. (2006). Names and similarities on the Web: Fact extraction in the fast lane. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL, pp. 809–816, Sydney, Australia. find
  124. Patwardhan, S., & Pedersen, T. (2006). Using wordnet-based context vectors to estimate the semantic relatedness of concepts. In Proceedings of the Workshop on Making Sense of Sense at the 11th Conference of the European Chapter of the Association for Computational Linguistics (EACL-2006), pp. 1–8. find
  125. Pedersen, T. (2006). Unsupervised corpus-based methods for WSD. In Word Sense Disambiguation: Algorithms and Applications, pp. 133–166. Springer. find
  126. Pennacchiotti, M., Cao, D. D., Basili, R., Croce, D., & Roth, M. (2008). Automatic induction of FrameNet lexical units. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing (EMNLP-08), pp. 457–465, Honolulu, Hawaii. find our card
  127. Pennacchiotti, M., & Pantel, P. (2006). Ontologizing semantic relations. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, pp. 793–800. Association for Computational Linguistics. find
  128. Pereira, F., Tishby, N., & Lee, L. (1993). Distributional clustering of English words. In Proceedings of the 31st Annual Meeting on Association for Computational Linguistics, pp. 183–190. find our card
  129. Porter, M. (1980). An algorithm for suffix stripping. Program, 14 (3), 130–137. find
  130. Rabin, M. O. (1981). Fingerprinting by random polynomials. Tech. rep., Center for research in Computing technology, Harvard University. Technical Report TR-15-81. find
  131. Rapp, R. (2003). Word sense discovery based on sense descriptor dissimilarity. In Proceedings of the Ninth Machine Translation Summit, pp. 315–322. find
  132. Ravichandran, D., Pantel, P., & Hovy, E. (2005). Randomized algorithms and nlp: using locality sensitive hash function for high speed noun clustering. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL ’05), pp. 622–629, Morristown, NJ. Association for Co… find
  133. Resnick, P., Iacovou, N., Suchak, M., Bergstrom, P., & Riedl, J. (1994). Grouplens: An open architecture for collaborative filtering of netnews. In Proceedings of the ACM 1994 Conference on Computer Supported Cooperative Work, pp. 175–186. ACM Press. find
  134. Resnik, P. (1995). Using information content to evaluate semantic similarity in a taxonomy. In Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI-95), pp. 448–453, San Mateo, CA. Morgan Kaufmann. find our card
  135. Rosario, B., & Hearst, M. (2001). Classifying the semantic relations in noun-compounds via a domain-specific lexical hierarchy. In Proceedings of the 2001 Conference on Empirical Methods in Natural Language Processing (EMNLP-01), pp. 82–90. find our card
  136. Rosario, B., Hearst, M., & Fillmore, C. (2002). The descent of hierarchy, and selection in relational semantics. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL-02), pp. 247–254. find
  137. Rosch, E., & Lloyd, B. (1978). Cognition and Categorization. Lawrence Erlbaum, Hillsdale, NJ. find
  138. Ruge, G. (1997). Automatic detection of thesaurus relations for information retrieval applications. In Freksa, C., Jantzen, M., & Valk, R. (Eds.), Foundations of Computer Science, pp. 499–506. Springer. find
  139. Sahami, M., Dumais, S., Heckerman, D., & Horvitz, E. (1998). A Bayesian approach to filtering junk e-mail. In Proceedings of the AAAI-98 Workshop on Learning for Text Categorization. find
  140. Sahlgren, M. (2005). An introduction to random indexing. In Proceedings of the Methods and Applications of Semantic Indexing Workshop at the 7th International Conference on Terminology and Knowledge Engineering (TKE), Copenhagen, Denmark. find
  141. Sahlgren, M. (2006). The Word-Space Model: Using distributional analysis to represent syntagmatic and paradigmatic relations between words in high-dimensional vector spaces. Ph.D. thesis, Department of Linguistics, Stockholm University. find
  142. Salton, G. (1971). The SMART retrieval system: Experiments in automatic document processing. Prentice-Hall, Upper Saddle River, NJ. find
  143. Salton, G., & Buckley, C. (1988). Term-weighting approaches in automatic text retrieval. Information Processing and Management, 24 (5), 513–523. find
  144. Salton, G., Wong, A., & Yang, C.-S. (1975). A vector space model for automatic indexing. Communications of the ACM, 18 (11), 613–620. find
  145. Sarawagi, S., & Kirpal, A. (2004). Efficient set joins on similarity predicates. In Proceedings of the 2004 ACM SIGMOD International Conference on Management of Data (SIGMOD ’04), pp. 743–754, New York, NY. ACM. find
  146. Scholkopf, B., Smola, A. J., & Muller, K.-R. (1997). Kernel principal component analysis. In Proceedings of the International Conference on Artificial Neural Networks (ICANN-1997), pp. 583–588, Berlin. find
  147. Schütze, H. (1998). Automatic word sense discrimination. Computational Linguistics, 24 (1), 97–124. find
  148. Schütze, H., & Pedersen, J. (1993). A vector model for syntagmatic and paradigmatic relatedness. In Making Sense of Words: Proceedings of the Conference, pp. 104–113, Oxford, England. find
  149. Sebastiani, F. (2002). Machine learning in automated text categorization. ACM Computing Surveys (CSUR), 34 (1), 1–47. find
  150. Shannon, C. (1948). A mathematical theory of communication. Bell System Technical Journal, 27, 379–423, 623–656. find
  151. Singhal, A., Salton, G., Mitra, M., & Buckley, C. (1996). Document length normalization. Information Processing and Management, 32 (5), 619–633. find
  152. Smith, E., Osherson, D., Rips, L., & Keane, M. (1988). Combining prototypes: A selective modification model. Cognitive Science, 12 (4), 485–527. find
  153. Snow, R., Jurafsky, D., & Ng, A. Y. (2006). Semantic taxonomy induction from heterogenous evidence. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the ACL, pp. 801–808. find
  154. Spärck Jones, K. (1972). A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation, 28 (1), 11–21. find
  155. Spearman, C. (1904). “General intelligence,” objectively determined and measured. American Journal of Psychology, 15, 201–293. find
  156. Sproat, R., & Emerson, T. (2003). The first international Chinese word segmentation bakeoff. In Proceedings of the Second SIGHAN Workshop on Chinese Language Processing, pp. 133–143, Sapporo, Japan. find
  157. Stone, P. J., Dunphy, D. C., Smith, M. S., & Ogilvie, D. M. (1966). The General Inquirer: A Computer Approach to Content Analysis. MIT Press, Cambridge, MA. find
  158. Tan, B., & Peng, F. (2008). Unsupervised query segmentation using generative language models and Wikipedia. In Proceeding of the 17th international conference on World Wide Web (WWW ’08), pp. 347–356, New York, NY. ACM. find
  159. Tellex, S., Katz, B., Lin, J., Fern, A., & Marton, G. (2003). Quantitative evaluation of passage retrieval algorithms for question answering. In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pp. 41–47. find
  160. Tucker, L. R. (1966). Some mathematical notes on three-mode factor analysis. Psychometrika, 31 (3), 279–311. find
  161. Turney, P. D. (2001). Mining the Web for synonyms: PMI-IR versus LSA on TOEFL. In Proceedings of the Twelfth European Conference on Machine Learning (ECML-01), pp. 491–502, Freiburg, Germany. find
  162. Turney, P. D. (2005). Measuring semantic similarity by latent relational analysis. In Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence (IJCAI-05), pp. 1136–1141, Edinburgh, Scotland. find
  163. Turney, P. D. (2006). Similarity of semantic relations. Computational Linguistics, 32 (3), 379–416. find our card
  164. Turney, P. D. (2007). Empirical evaluation of four tensor decomposition algorithms. Tech. rep., Institute for Information Technology, National Research Council of Canada. Technical Report ERB-1152. find
  165. Turney, P. D. (2008a). The latent relation mapping engine: Algorithm and experiments. Journal of Artificial Intelligence Research, 33, 615–655. find
  166. Turney, P. D. (2008b). A uniform approach to analogies, synonyms, antonyms, and associations. In Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pp. 905–912, Manchester, UK. find
  167. Turney, P. D., & Littman, M. L. (2003). Measuring praise and criticism: Inference of semantic orientation from association. ACM Transactions on Information Systems, 21 (4), 315–346. find
  168. Turney, P. D., & Littman, M. L. (2005). Corpus-based learning of analogies and semantic relations. Machine Learning, 60 (1–3), 251–278. find
  169. Turney, P. D., Littman, M. L., Bigham, J., & Shnayder, V. (2003). Combining independent modules to solve multiple-choice synonym and analogy problems. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP-03), pp. 482–489, Borovets, Bulgaria. find
  170. Van de Cruys, T. (2009). A non-negative tensor factorization model for selectional preference induction. In Proceedings of the Workshop on Geometric Models for Natural Language Semantics (GEMS-09), pp. 83–90, Athens, Greece. van Rijsbergen, C. J. (2004). The Geometry of Information Retrieval. find
  171. Cambridge University Press, Cambridge, UK. van Rijsbergen, C. J. (1979). Information Retrieval. Butterworths. find
  172. Veale, T. (2003). The analogical thesaurus. In Proceedings of the 15th Innovative Applications of Artificial Intelligence Conference (IAAI 2003), pp. 137–142, Acapulco, Mexico. find
  173. Veale, T. (2004). WordNet sits the SAT: A knowledge-based approach to lexical analogy. In Proceedings of the 16th European Conference on Artificial Intelligence (ECAI 2004), pp. 606–612, Valencia, Spain. find
  174. Vozalis, E., & Margaritis, K. (2003). Analysis of recommender systems algorithms. In Proceedings of the 6th Hellenic European Conference on Computer Mathematics and its Applications (HERCMA-2003), Athens, Greece. find
  175. Vyas, V., & Pantel, P. (2009). Semi-automatic entity set refinement. In Proceedings of NAACL-09, Boulder, CO. find
  176. Weaver, W. (1955). Translation. In Locke, W., & Booth, D. (Eds.), Machine Translation of Languages: Fourteen Essays. MIT Press, Cambridge, MA. find
  177. Weeds, J., Weir, D., & McCarthy, D. (2004). Characterising measures of lexical distributional similarity. In Proceedings of the 20th International Conference on Computational Linguistics (COLING ’04), pp. 1015–1021. Association for Computational Linguistics. find
  178. Wei, X., Peng, F., & Dumoulin, B. (2008). Analyzing web text association to disambiguate abbreviation in queries. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’08), pp. 751–752, New York, NY. ACM. find
  179. Wen, J.-R., Nie, J.-Y., & Zhang, H.-J. (2001). Clustering user queries of a search engine. In Proceedings of the 10th International Conference on World Wide Web (WWW ’01), pp. 162–168, New York, NY. ACM. find
  180. Widdows, D. (2004). Geometry and Meaning. Center for the Study of Language and Information, Stanford, CA. find
  181. Widdows, D., & Ferraro, K. (2008). Semantic vectors: A scalable open source package and online technology management application. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC 2008), pp. 1183–1190. find
  182. Witten, I. H., & Frank, E. (2005). Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, San Francisco. find
  183. Wittgenstein, L. (1953). Philosophical Investigations. Blackwell. Translated by G.E.M. Anscombe. find
  184. Wolfe, M. B. W., Schreiner, M. E., Rehder, B., Laham, D., Foltz, P. W., Kintsch, W., & Landauer, T. K. (1998). Learning from text: Matching readers and texts by latent semantic analysis. Discourse Processes, 25, 309–336. find
  185. Yang, Y. (1999). An evaluation of statistical approaches to text categorization. Information Retrieval, 1 (1), 69–90. find
  186. Yuret, D., & Yatbaz, M. A. (2009). The noisy channel model for unsupervised word sense disambiguation. Computational Linguistics. Under review. find
  187. Zamir, O., & Etzioni, O. (1999). Grouper: a dynamic clustering interface to Web search results. Computer Networks: The International Journal of Computer and Telecommunications Networking, 31 (11), 1361–1374. find
  188. Zhao, Y., & Karypis, G. (2002). Evaluation of hierarchical clustering algorithms for document datasets. In Proceedings of the Eleventh International Conference on Information and Knowledge Management, pp. 515–524, McLean, Virginia. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/turney-2010-frequency-meaning-vector-space-models/