Elamur Research⌕
☰
home / papers / 2006 / Broad-Coverage Sense Disambiguation and Information Extraction with a Supersense Sequence Tagger

Broad-Coverage Sense Disambiguation and Information Extraction with a Supersense Sequence Tagger

Summary

The paper argues that named entity recognition uses too few categories to be broadly useful, while fine-grained word sense disambiguation over Wordnet synsets has too many classes and too little training data to beat simple baselines. The authors propose a middle ground. They tag nouns and verbs with the 41 Wordnet lexicographer classes, called supersenses. This tagset still gives partial sense disambiguation and also contains NER-style classes such as person, location, group and time. They treat supersense tagging as sequence labeling and train a discriminative HMM in the style of Collins (2002) on the hand-annotated Semcor corpus. They evaluate on held-out Semcor data and on the Senseval-3 English all-words data. The tagger clearly beats the strong first-sense heuristic on both. According to the authors, this is the first convincing gain over that baseline in the realistic all-words setting. The work shows that coarse but meaningful semantic labels can be learned robustly with standard tagging machinery, which unifies extended NER and WSD in a single tool. The tagger was released as free software.

Key points

  • The tagset has 41 supersenses from Wordnet 2.0: 26 for nouns and 15 for verbs. For example, the 10 synsets of the noun "box" collapse into 6 supersenses.
  • Compared with the first-sense baseline, the tagger gains 10.71 points on Semcor (31.19% error reduction) and 6.45 points on Senseval-3 (17.96% error reduction).
  • For context, the Senseval-3 all-words task had a baseline of 62.40%. Only 4 of 26 participating systems beat it, and the best reached 65.2% F-score.
  • On the NER-like classes in Semcor (person, group, location, time), which include common nouns as well as names, the average F-score is 82.46%.
  • On Senseval-3, the tagger loses to the baseline on noun.person (73.06 vs 83.97 F) and on verb.cognition. The main reason is an annotation mismatch: Semcor tags named entities, but Senseval does not.

Interesting ideas

  • I1. Use an existing coarse layer of a lexicon as the label set. Grouping senses by Wordnet's lexicographer files removes hard, subtle distinctions while keeping the major ones. A similar coarse grouping in another language's wordnet or lexicon could make sense tagging trainable with limited data.
  • I2. Merge NER and WSD into one tagging task. A single sequence tagger can label both proper and common nouns, plus verbs, with semantic classes. The result is an "extended NER" that also marks frequently neglected types such as artifacts, or verb classes that hint at relations between entities.
  • I3. Model label dependencies along the sentence. Most WSD systems label each word in isolation. A sequence model can instead use neighboring senses, for example a coordinated motion verb and communication verb. This is cheap to try wherever semantic labels are likely correlated within a sentence.
  • I4. Always measure against the first-sense heuristic and report error reduction. Comparing a random-sense baseline with a most-frequent-sense baseline shows how strong the frequency prior really is. Error-reduction figures make gains comparable across datasets.
  • I5. Check that training and test annotation guidelines match. The drop on noun.person came from Senseval leaving named entities unannotated. Per-class breakdowns help expose this kind of guideline mismatch, rather than mistaking it for model failure.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Word sense disambiguation · Information extraction

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{ciaramita2006broad,
  title = {{Broad-Coverage Sense Disambiguation and Information Extraction with a Supersense Sequence Tagger}},
  author = {Ciaramita, Massimiliano and Altun, Yasemin},
  booktitle = {Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing},
  year = {2006},
  doi = {10.3115/1610075.1610158},
  url = {https://doi.org/10.3115/1610075.1610158}
}

References

29 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Y. Altun, T. Hofmann, and M. Johnson. 2003. Discriminative Learning for Label Sequences via Boosting. In Proceedings of NIPS 2003. find
  2. T. Brants. 2002. TnT - A Statistical Part-of-Speech Tagger. In Proceedings of ANLP 2000. find our card
  3. X. Carreras, L. Marquez, and L. Padro. 2002. Named Entity Extraction Using AdaBoost. In Proceedings of CONLL 2002. find
  4. M. Ciaramita and M. Johnson. 2003. Supersense Tagging of Unknown Nouns in WordNet. In Proceedings of EMNLP 2003. find our card
  5. M. Ciaramita, T. Hofmann, and M. Johnson. 2003. Hierarchical Semantic Classification: Word Sense Disambiguation with World Knowledge. In Proceedings of IJCAI 2003. find
  6. M. Ciaramita, S. Sloman, M. Johnson, and E. Upfal. 2005. Hierarchical Preferences in a Broad-Coverage Lexical Taxonomy. In Proceedings of CogSci 2005. find
  7. M. Collins. 2002. Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms. In Proceedings of EMNLP 2002, pages 1–8. find our card
  8. J. Curran. 2005. Supersense Tagging of Unknown Nouns Using Semantic Similarity. In Proceedings of ACL 2005, pages 26–33. find our card
  9. C. de Loupy, M. El-Beze, and P.F. Marteau. 1998. Word Sense Disambiguation Using HMM Tagger. In Proceedings of LREC 1998, pages 1255–1258. find
  10. B. Decadt, V. Hoste, W. Daelemans, and A. van der Bosch. 2004. GAMBL, Genetic Algorithm Optimization of Memory-Based WSD. In Proceedings of SENSEVAL-3/ACL 2004. find
  11. S. Dingare, M. Nissim, J. Finkel, C. Manning, and C. Grover. 2005. A System for Identifying Named Entities in Biomedical Text: How Results from Two Evaluations Reflect on Both the System and the Evaluations. Comparative and Functional Genomics, 6:77–85. find
  12. C. Fellbaum, editor. 1998. WordNet: An Electronic Lexical Database. MIT Press, Cambridge. find
  13. R. Florian, A. Ittycheriah, H. Jing, and T. Zhang. 2003. Named Entity Extraction through Classifier Combination. In Proceedings of CONLL 2003. find
  14. W. Gale, K. Church, and D. Yarowsky. 1992. One Sense per Discourse. In Proceedings of the DARPA Workshop on Speech and Natural Language. find
  15. J. Kazama, T. Makino, Y. Ohta, and J. Tsujii. 2002. Tuning Support Vector Machines for Biomedical Named Entity Recognition. In Proceedings of the Workshop on Natural Language Processing in the Biomedical Domain (ACL 2002). find
  16. T. Koo and M. Collins. 2005. Hidden-Variable Models for Discriminative Reranking. In Proceedings of EMNLP 2005. find
  17. H. Kuc̆era and W. Francis. 1967. Computational Analysis of Present-Day American English. Brown University Press, Providence, RI. find
  18. J. Lafferty, A. McCallum, and F. Pereira. 2001. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In Proceedings of ICML 2001, pages 282–289. find
  19. A. McCallum, D. Freitag, and F. Pereira. 2000. Maximum Entropy Markov Models for Information Extraction and Segmentation. In Proceedings of ICML 2000, pages 591–598. find
  20. D. McCarthy, R. Koeling, and J. Carroll. 2004. Finding Predominant Senses in Untagged Text. In Proceedings of ACL 2004. find our card
  21. R. Mihalcea and E. Faruque. 2004. SenseLearner: Minimally Supervised Word Sense Disambiguation for All Words in Open Text. In Proceedings of SENSEVAL-3/ACL 2004. find
  22. G.A. Miller, C. Leacock, T. Randee, and R. Bunker. 1993. A Semantic Concordance. In Proceedings of the 3 DARPA Workshop on Human Language Technology, pages 303–308. find
  23. A. Molina, F. Pla, and E. Segarra. 2002. A Hidden Markov Model Approach to Word Sense Dsi-ambiguation. In Proceedings of IBERAMIA 2002. find
  24. A. Molina, F. Pla, and E. Segarra. 2004. WSD System Based on Specialized Hidden Markov Model (upv-shmm-eaw). In Proceedings of SENSEVAL-3/ACL 2004. find
  25. Y. Ohta, Y. Tateisi, J. Kim, H. Mima, and J. Tsujii. 2002. The GENIA Corpus: An Annotated Research Abstract Corpus in the Molecular Biology Domain. In Proceedings of HLT 2002. find
  26. B. Rosario and M. Hearst. 2004. Classifying Semantic Relations in Bioscience Text. In Proceedings of ACL 2004). find
  27. F. Segond, A. Schiller, G. Grefenstette, and J.P. Chanod. 1997. An Experiment in Semantic Tagging Using Hidden Markov Model. In Proceedings of the Workshop on Automatic Information Extraction and Building of Lexical Semantic Resources (ACL/EACL 1997), pages 78–81. find
  28. F. Sha and F. Pereira. 2003. Shallow Parsing with Conditional Random Fields. In Proceedings of HLT-NAACL 2003, pages 213–220. find
  29. B. Snyder and M. Palmer. 2004. The english All-Words Tasks. In Proceedings of SENSEVAL-3/ACL 2004. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ciaramita-2006-broad-coverage-sense-disambiguation-information/