Elamur Research⌕
☰
home / papers / 2008 / Automatic induction of FrameNet lexical units

Automatic induction of FrameNet lexical units

Summary

FrameNet has limited coverage, with only about 10,000 lexical units (LUs). This has repeatedly undermined its use in QA and RTE systems. The authors address LU induction: automatically assigning words absent from FrameNet to the correct frame(s) among roughly 800 candidates.

They propose two unsupervised models and a hybrid:

  • Distributional model: represents each frame as a frequency-weighted centroid of its LUs' co-occurrence vectors and ranks frames by cosine similarity. It is tested with word-window, syntax-based and mixed (dependency-path) spaces.
  • WordNet-based model: builds per-frame hyponymy sub-graphs for nouns, verbs and adjectives, scoring nouns with conceptual density.
  • Combined model: merges the two.

Evaluation used leave-one-out on the FrameNet gold standard. The WordNet approach is more precise but covers fewer words. Distributional spaces cover nearly any word and do not require WordNet, so they suit low-resource languages. The combination gives the best balance between accuracy and coverage, beating earlier systems. Simple word-based spaces outperformed richer syntactic ones. The authors argue the results are good enough to support semi-automatic extension of FrameNet and bootstrapping of FrameNets in new languages.

Key points

  • Resource scale: FrameNet has 795 frames, about 10,000 LUs and 135,000 annotated BNC sentences, compared with roughly 210,000 entries in WordNet 3.0.
  • Comparison with Detour (Burchardt et al., 2005): Detour reports 0.38 best-1 accuracy at 87% coverage. The combined model surpasses it by about 5% in accuracy and 8% in coverage, and also supports best-k ranking.
  • Comparison with Johansson & Nugues (2007), an SVM over WordNet features: at 80% coverage their system reaches about 0.42 accuracy, roughly 10 points below the best WordNet-based model here. At 90% coverage it falls below 0.10.
  • Frequency threshold: restricting the distributional model to LUs with at least 200 corpus occurrences raises accuracy by 0.12 but lowers coverage to 57%.
  • SemEval-2007 test: on 24 unknown LUs, WNet-full scores 0.25 (best-1) and 0.69 (best-10) at 67% coverage. Many apparent errors are plausible frames, so reported accuracies should be read as lower bounds.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Semantic role labeling

Cited by

26 papers of our library cite it (of the 140673 whose bibliographies we read); 9 of them have a page here, 17 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{pennacchiotti2008automatic,
  title = {{Automatic induction of FrameNet lexical units}},
  author = {Pennacchiotti, Marco and Cao, Diego De and Basili, Roberto and Croce, Danilo and Roth, Michael},
  booktitle = {Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing},
  year = {2008},
  doi = {10.3115/1613715.1613773},
  url = {https://doi.org/10.3115/1613715.1613773}
}

References

23 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. E. Agirre and G. Rigau. 1996. Word Sense Disambiguation using Conceptual Density. In Proceedings of COLING-96, Copenhagen, Denmark. find
  2. Collin F. Baker, Charles J. Fillmore, and John B. Lowe. 1998. The Berkeley FrameNet project. In Proceedings of COLING-ACL, Montreal, Canada. find our card
  3. Collin Baker, Michael Ellsworth, and Katrin Erk. 2007. SemEval-2007 Task 19: Frame Semantic Structure Extraction. In Proceedings of the Fourth International Workshop on Semantic Evaluations (SemEval-2007), pages 99–104, Prague, Czech Republic, June. find our card
  4. Roy Bar-Haim, Idan Szpektor, and Oren Glickman. 2005. Definition and Analysis of Intermediate Entailment Levels. In ACL-05 Workshop on Empirical Modeling of Semantic Equivalence and Entailment, Ann Arbor, Michigan. find
  5. R. Basili, M. Cammisa, and F.M. Zanzotto. 2004. A semantic similarity measure for unsupervised semantic disambiguation. In Proceedings of LREC-04, Lisbon, Portugal. find
  6. Alexander Budanitsky and Graeme Hirst. 2006. Evaluating WordNet-based measures of semantic distance. Computational Linguistics, 32(1):13–47. find
  7. Aljoscha Burchardt and Anette Frank. 2006. Approximating Textual Entailment with LFG and FrameNet Frames. In Proceedings of PASCAL RTE2 Workshop. find
  8. Aljoscha Burchardt, Katrin Erk, and Anette Frank. 2005. A WordNet Detour to FrameNet. In Sprachtech-nologie, mobile Kommunikation und linguistische Re-sourcen, volume 8 of Computer Studies in Language and Speech. Peter Lang, Frankfurt/Main. find
  9. Aljoscha Burchardt, Katrin Erk, Anette Frank, Andrea Kowalski, Sebastian Padó, and Manfred Pinkal. 2006. The SALSA corpus: a German corpus resource for lexical semantics. In Proceedings of LREC, Genova, Italy. find our card
  10. Ted Dunning. 1993. Accurate methods for the statistics of surprise and coincidence. Computational Linguistics, 18(1):61–74. find our card
  11. Charles J. Fillmore. 1985. Frames and the semantics of understanding. Quaderni di Semantica, 4(2):222–254. find
  12. K. Garoufi. 2007. Towards a better understanding of applied textual entailment: Annotation and evaluation of the rte-2 dataset. M.Sc. thesis, saarland university. find
  13. Zellig Harris. 1964. Distributional structure. In Jerrold J. Katz and Jerry A. Fodor, editors, The Philosophy of Linguistics, New York. Oxford University Press. find
  14. Richard Johansson and Pierre Nugues. 2007. Using WordNet to extend FrameNet coverage. In Proceedings of the Workshop on Building Frame-semantic Resources for Scandinavian and Baltic Languages, at NODALIDA, Tartu, Estonia, May 24. find
  15. Dekang Lin. 1998. Automatic retrieval and clustering of similar word. In Proceedings of COLING-ACL, Montreal, Canada. find
  16. Sebastian Padó. 2007. Cross-Lingual Annotation Projection Models for Role-Semantic Information. Saarland University. find
  17. M. Palmer, D. Gildea, and P. Kingsbury. 2005. The Proposition Bank: An Annotated Corpus of Semantic Roles. Computational Linguistics, 31(1). find our card
  18. Ted Pedersen, Siddharth Patwardhan, and Jason Michelizzi. 2004. WordNet::Similarity - Measuring the Relatedness of Concept. In Proc. of 5th NAACL, Boston, MA. find
  19. Magnus Sahlgren. 2006. The Word-Space Model. Department of Linguistics, Stockholm University. find
  20. G. Salton, A. Wong, and C. Yang. 1975. A vector space model for automatic indexing. Communications of the ACM, 18:613620. find
  21. Hinrich Schütze. 1998. Automatic Word Sense Discrimination. Computational Linguistics, 24(1):97–124. find our card
  22. Dan Shen and Mirella Lapata. 2007. Using semantic roles to improve question answering. In Proceedings of EMNLP-CoNLL, pages 12–21, Prague. find
  23. C. Subirats and M. Petruck. 2003. Surprise! Spanish FrameNet! In Proceedings of the Workshop on Frame Semantics at the XVII. International Congress of Linguists, Prague. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/pennacchiotti-2008-automatic-induction-framenet-lexical-units/