Elamur Research⌕
☰
home / papers / 2008 / A Uniform Approach to Analogies, Synonyms, Antonyms, and Associations

A Uniform Approach to Analogies, Synonyms, Antonyms, and Associations

Paper · 2008 · Peter Turney

Summary

Turney proposes treating four lexical-semantic tasks with one method: recognizing analogies, synonyms, antonyms and associations. These are usually tackled by separate, specialised algorithms. The central move is to reframe each relation as an analogy. For example, a pair counts as antonymous if it relates the way black:white does, and as associated if it resembles doctor:hospital. Every task thus becomes supervised classification of word pairs.

The resulting algorithm, PairClass, works in four steps:

  1. It expands each pair morphologically.
  2. It collects short phrases containing both words from a web corpus of about 5 × 10^10 words.
  3. It turns those phrases into generalized wildcard patterns and keeps the most widely shared ones.
  4. It feeds log-frequency feature vectors to an SVM.

The same configuration, with no per-task tuning, was applied to:

  • SAT analogy questions
  • TOEFL synonym questions
  • ESL synonym/antonym questions
  • similar/associated/both word pairs from cognitive psychology

PairClass is competitive but does not beat the strongest specialised systems. Its value lies in showing that a single, purely corpus-based relational classifier can cover several semantic phenomena without a lexicon such as WordNet.

Key points

  • Accuracy by task:
    • SAT analogies: 52.1% (best previous 56.1%, human 57.0%, baseline 20%)
    • TOEFL synonyms: 76.2% (best previous 97.5%, human 64.5%, baseline 25%)
    • ESL synonyms vs. antonyms: 75.0% (baseline 65.4%)
    • similar/associated/both: 77.1% (baseline 33.3%)
  • Phrase template and patterns: phrases follow "[0–1 words] X [0–3 words] Y [0–1 words]" and are retrieved with the Wumpus search engine. Morphology is handled by morpha/morphg. Each phrase of n words yields 2^(n−2) patterns.
  • Feature selection and weighting: only the top kN patterns are kept, where N is the number of word pairs and k = 20, ranked by how many pairs produce each pattern. Feature values are log(f+1), and each vector is normalized to unit length.
  • Classifier: Weka's SMO SVM with an RBF kernel, which outputs class probabilities via logistic fitting. Unlike Turney (2006), it uses no SVD smoothing, no lexicon, and no nearest-neighbour step.
  • Main limitation: the method depends on a very large corpus, because phrases containing both words of a pair are rare.

Interesting ideas

  • I1. Defining relations by exemplar pairs. A relation can be specified through example pairs ("relates like X:Y") instead of a hand-built resource. This makes it cheap to add new relation types or move to another language, such as Ukrainian, where lexicons like WordNet are thinner.
  • I2. Feature budget that scales with the data. Keeping k features per input item, rather than a fixed number, grows the feature space with the dataset. It is a simple rule that reportedly needs little tuning.
  • I3. Faking a negative example. With only one labelled positive (an SAT stem), the stem of another random question is borrowed as a pseudo-negative. This trick can bootstrap tiny classification problems in other settings.
  • I4. Generalizing context by wildcard subsets. Every subset of context words is replaced with wildcards to produce patterns. Ranking those patterns by how many pairs share them gives an unsupervised way to discover relational cues that generalize across pairs.
  • I5. Probabilities over similarity scores. Producing class probabilities instead of cosine scores makes the outputs easier to reuse in downstream pipelines.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

📚 Shelves 4 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works

🧭 Topics 1 Lexical semantics & word embeddings · 40 works

Cite

Cite as Peter Turney. 2008. A Uniform Approach to Analogies, Synonyms, Antonyms, and Associations. Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{turney2008uniform,
  title = {{A Uniform Approach to Analogies, Synonyms, Antonyms, and Associations}},
  author = {Peter Turney},
  booktitle = {Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008)},
  year = {2008},
  doi = {10.3115/1599081.1599195},
  url = {https://doi.org/10.3115/1599081.1599195}
}

References

28 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Breiman, Leo. 1996. Bagging predictors. Machine Learning, 24(2):123–140.
  2. Büttcher, Stefan and Charles Clarke. 2005. Efficiency vs. effectiveness in terabyte-scale information retrieval. In Proceedings of the 14th Text REtrieval Conference (TREC 2005), Gaithersburg, MD. find
  3. Chiarello, Christine, Curt Burgess, Lorie Richards, and Alma Pollock. 1990. Semantic and associative priming in the cerebral hemispheres: Some words do, some words don’t ... sometimes, some places. Brain and Language, 38:75–104. find
  4. Falkenhainer, Brian, Kenneth D. Forbus, and Dedre Gentner. 1989. The structure-mapping engine: Algorithm and examples. Artificial Intelligence, 41(1):1–63. find
  5. Fellbaum, Christiane, editor. 1998. WordNet: An Electronic Lexical Database. MIT Press. find
  6. Girju, Roxana, Preslav Nakov, Vivi Nastase, Stan Szpakowicz, Peter Turney, and Deniz Yuret. 2007. Semeval-2007 task 04: Classification of semantic relations between nominals. In SemEval 2007, pages 13–18, Prague, Czech Republic.
  7. Hirst, Graeme and David St-Onge. 1998. Lexical chains as representations of context for the detection and correction of malapropisms. In Fellbaum, Christiane, editor, WordNet: An Electronic Lexical Database, pages 305–332. MIT Press. find
  8. Hofstadter, Douglas. 2007. I Am a Srange Loop. Basic Books. find
  9. Jiang, Jay J. and David W. Conrath. 1997. Semantic similarity based on corpus statistics and lexical taxonomy. In ROCLING X, pages 19–33, Tapei, Taiwan. find
  10. Landauer, Thomas K. and Susan T. Dumais. 1997. A solution to Plato’s problem: The latent semantic analysis theory of the acquisition, induction, and representation of knowledge. Psychological Review, 104(2):211–240. find
  11. Lepage, Yves and Etienne Denoual. 2005. Purest ever example-based machine translation: Detailed presentation and assessment. Machine Translation, 19(3):251–282. find
  12. Lepage, Yves. 1998. Solving analogies on words: An algorithm. In Proceedings of the 36th Annual Conference of the Association for Computational Linguistics, pages 728–735. find
  13. Lesk, Michael E. 1969. Word-word associations in document retrieval systems. American Documentation, 20(1):27–38. find
  14. Lin, Dekang, Shaojun Zhao, Lijuan Qin, and Ming Zhou. 2003. Identifying synonyms among distributionally similar words. In IJCAI-03, pages 1492– 1493. find
  15. Lund, Kevin, Curt Burgess, and Ruth Ann Atchley. 1995. Semantic and associative priming in high-dimensional semantic space. In Proceedings of the 17th Annual Conference of the Cognitive Science Society, pages 660–665. find
  16. Minnen, Guido, John Carroll, and Darren Pearce. 2001. Applied morphological processing of English. Natural Language Engineering, 7(3):207–223. find
  17. Minsky, Marvin. 1986. The Society of Mind. Simon & Schuster, New York, NY. find
  18. Nastase, Vivi and Stan Szpakowicz. 2003. Exploring noun-modifier semantic relations. In Fifth International Workshop on Computational Semantics (IWCS-5), pages 285–301, Tilburg, The Netherlands. find
  19. Platt, John C. 1998. Fast training of support vector machines using sequential minimal optimization. In Advances in Kernel Methods: Support Vector Learning, pages 185–208. MIT Press Cambridge, MA, USA. find
  20. Reitman, Walter R. 1965. Cognition and Thought: An Information Processing Approach. John Wiley and Sons, New York, NY. find
  21. Resnik, Philip. 1995. Using information content to evaluate semantic similarity in a taxonomy. In IJCAI-95, pages 448–453, San Mateo, CA. Morgan Kaufmann.
  22. Rosario, Barbara and Marti Hearst. 2001. Classifying the semantic relations in noun-compounds via a domain-specific lexical hierarchy. In EMNLP-01, pages 82–90.
  23. Turney, Peter D. and Michael L. Littman. 2005. Corpus-based learning of analogies and semantic relations. Machine Learning, 60(1–3):251–278. find
  24. Turney, Peter D., Michael L. Littman, Jeffrey Bigham, and Victor Shnayder. 2003. Combining independent modules to solve multiple-choice synonym and analogy problems. In RANLP-03, pages 482–489, Borovets, Bulgaria. find
  25. Turney, Peter D. 2001. Mining the Web for synonyms: PMI-IR versus LSA on TOEFL. In Proceedings of the Twelfth European Conference on Machine Learning, pages 491–502, Berlin. Springer. find
  26. Turney, Peter D. 2006. Similarity of semantic relations. Computational Linguistics, 32(3):379–416.
  27. Veale, Tony. 2004. WordNet sits the SAT: A knowledge-based approach to lexical analogy. In Proceedings of the 16th European Conference on Artificial Intelligence (ECAI 2004), pages 606–612, Valencia, Spain. find
  28. Witten, Ian H. and Eibe Frank. 1999. Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, San Francisco. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/turney-2008-uniform-approach-analogies-synonyms-antonyms/