📄 UTD-SRL: A Pipeline Architecture for Extracting Frame Semantic Structures

Summary

The paper presents UTD-SRL, a system built for the SemEval-2007 task of extracting frame semantic structures based on FrameNet (FN). In this task a system must find target words, choose the frame each one evokes, and locate and label its frame elements (FE), including grammatical function (GF) and phrase type. Extending their Senseval-3 work, the authors break the problem into four classification steps: target word frame disambiguation (TWFD), FE boundary detection (FEBD), GF label classification (GFLC) and FE label classification (FELC). The steps are chained so that later classifiers can use earlier predictions, such as the predicted frame, candidate FE lists and GF labels, as features. Sentences are parsed with the Collins parser and tagged by an in-house named entity recognizer. For each step, both SVM and Maximum Entropy (ME) models are trained, and the better one is chosen on held-out data. On the test set the system reached competitive precision, but low recall pulled down F1. The work shows how a pipeline of feature-sharing classifiers can be applied to full frame-semantic parsing.

Key points

• Classifier inventory: 556 per-word multi-class classifiers for frame disambiguation, one binary classifier for FE boundaries, one multi-class classifier for GF labels (seven FN categories plus NULL), and 489 per-frame classifiers for FE labels.
• Validation results used 20% of the training data. Frame disambiguation reached 76.71% accuracy with SVM, against a 60.72% most-frequent-frame baseline. GFLC reached 96.00% (ME) and FELC 88.93% (ME), both measured with gold FE boundaries. FEBD (SVM) reached an F1 of about 79.80.
• On the test set, semantic dependency evaluation gave precision of roughly 50–57%, recall of about 27–30%, and F1 of about 35–39%, depending on scorer options.
• Frame detection on the test set reached precision of 69–80% and F1 of 52.71–61.35.
• The FRAME feature, produced by the TWFD step, improved boundary detection in every feature-selection run. The authors attribute the weak recall to imperfect mapping of FEs onto parse-tree constituents.