Maximum Entropy Models for FrameNet Classification

Summary

The paper tackles automatic semantic role labeling on FrameNet, where each annotated sentence has a target predicate whose frame elements must be assigned one of 120 semantic roles. Data are sparse, averaging about 30 tagged sentences per predicate. The authors build on Gildea and Jurafsky (2002), the only prior statistical classifier for FrameNet, and use the same training, development and test splits so the results are directly comparable. They replace that work's linearly interpolated probability model with a log-linear Maximum Entropy classifier. They also add two kinds of features: sentence-level syntactic patterns (the sequence of phrase types and logical functions across all elements) and the roles assigned to preceding elements. Viterbi search then selects the most probable role sequence for each sentence. On both gold and automatically identified boundaries, the system improves on the earlier work by more than 6%, a statistically significant gain (p<0.01). Learning curves indicate that classification would benefit from more data. Boundary identification, by contrast, appears to have plateaued and would need better models.

Key points

Data: The splits match Gildea & Jurafsky exactly: 36,993 training sentences (75,548 frame elements), 4,000 development sentences and 3,865 test sentences (7,899 frame elements). They are drawn from the June 2002 FrameNet release, which covers 1,462 predicates.
Model: The classifier has 12 feature sets totaling 203,443 binary feature functions. Training uses GIS in the YASMET package with Gaussian-prior smoothing, and the YASMET MEtagger performs the Viterbi decoding.
Classification accuracy: On automatically identified elements, the ME classifier reaches 81.5% accuracy, close to its result on hand-labeled boundaries. The earlier baseline reported 78.5%.
Boundary identification: The ME binary identifier scores F = .706, compared with .675 for Gildea & Jurafsky. On the combined identification-plus-labeling task, the scores are .576 versus .551, again in the ME model's favor.
Bottleneck: Identification is the weak point. Its upper bound on recall is 86.9%, because not every frame element matches a parse constituent.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Semantic role labeling

Cited by

21 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 15 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{fleischman2003maximum,
  title = {{Maximum Entropy Models for FrameNet Classification}},
  author = {Fleischman, Michael and Kwon, Namhee and Hovy, Eduard},
  booktitle = {Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing},
  year = {2003},
  doi = {10.3115/1119355.1119362},
  url = {https://doi.org/10.3115/1119355.1119362}
}

References

8 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

O. Bender, K. Macherey, F. J. Och, and H. Ney. 2003. Comparison of Alignment Templates and Maximum Entropy Models for Natural Language Processing. Proc. of EACL-2003. Budapest, Hungary. find
A. Berger, S. Della Pietra and V. Della Pietra, 1996. A Maximum Entropy Approach to Natural Language Processing. Computational Linguistics, vol. 22, no. 1. find our card
S. F. Chen and R. Rosenfeld. 1999. A Gaussian prior for smoothing maximum entropy models. Technical Report CMUCS -99-108, Carnegie Mellon University M. Collins. 1997. Three generative, lexicalized models for statistical parsing. Proc. of the 35 th Annual Meeting of the ACL. pages 16-23, Madrid, Spai… find
J. N. Darroch and D. Ratcliff. 1972. Generalized iterative scaling for log-linear models. Annals of Mathematical Statistics, 43:1470-1480. find
C. Fillmore 1976. Frame semantics and the nature of language. Annals of the New York Academy of Sciences: Conference on the Origin and Development of Language and Speech, Volume 280 (pp. 20-32). find
D. Gildea and D. Jurafsky. 2002. Automatic Labeling of Semantic Roles, Computational Linguistics, 28(3) 245-288 14. find our card
T. Mitchell. 1997. Machine Learning. McGraw-Hill International Editions, New York, NY. Pages 143-145. find
F.J. Och. 2002. Yet another maxent toolkit: YASMET. www-i6.informatik.rwth-aachen.de/Colleagues/och/. find

This page in Markdown → https://md.research.elamur.ai/free/en/papers/fleischman-2003-maximum-entropy-models-framenet-classification/

📄 Maximum Entropy Models for FrameNet Classification