Elamur Research⌕
☰
home / concepts / Semantic role labeling

🎭 Semantic role labeling

Concept · 30 key papers · all concepts

Semantic role labeling is the task of finding, for each predicate in a sentence, the phrases that act as its participants and labeling each with a role such as agent, theme, time or place. The result is a shallow picture of who did what to whom, plus when and where.

Also: SRL, shallow semantic parsing, predicate–argument structure labeling, thematic role labeling, frame-semantic parsing (FrameNet variant), QA-SRL (question-based variant)

What it is

Semantic role labeling (SRL) sits between syntax and full semantic interpretation. A parser can say that a noun phrase is the subject of a verb. It cannot say that the same participant keeps the same meaning across different syntactic realizations. Take "The window broke" and "John broke the window". The window is the broken thing in both sentences, although it is the subject in one and the object in the other. SRL captures this invariance by attaching role labels to the arguments of each predicate (Palmer et al. 2005). The linguistic roots go back to case grammar (Fillmore 1968) and to work on verb classes and alternations (Levin 1993).

Two annotation styles dominate. In the PropBank style, each verb sense has numbered core arguments plus general modifiers:

[ArgM-TMP Yesterday], [Arg0 Mary] [V sold] [Arg1 her old car] [Arg2 to a neighbor] [ArgM-LOC in Boston].

Arg0 roughly corresponds to a proto-agent, and Arg1 to a proto-patient or theme. Numbers above Arg1 carry no meaning shared across verbs. ArgM labels such as TMP, LOC or MNR can attach to any verb (Palmer et al. 2005).

In the FrameNet style, the word "sold" evokes the Commerce_sell frame, and the same sentence becomes:

[Time Yesterday], [Seller Mary] sold [Goods her old car] [Buyer to a neighbor] [Place in Boston].

FrameNet roles, called frame elements, are defined per conceptual situation rather than universally or per verb (Baker et al. 1998).

ResourceRole inventoryGranularity
PropBankArg0–Arg5, ArgM-*per verb sense
FrameNetframe elements (Seller, Goods…)per frame
VerbNetabout 20 thematic labels (early versions)per verb class

VerbNet groups verbs into Levin-style classes, and PropBank annotators used it as a consistency reference. At that time, VerbNet used about 20 thematic labels. Only around half of its verbs also occurred among the verbs of Penn TreeBank II (Palmer et al. 2005).

The key idea

SRL is usually decomposed into a pipeline:

Punyakanok and colleagues used the last four of these stages, with predicates given. Their pipeline ends with a global step solved by integer linear programming (Punyakanok et al. 2008). FrameNet parsing follows a parallel split into target, frame and argument identification (Das et al. 2014).

The classical statistical core is a role classifier conditioned on syntactic features of a candidate constituent:

r̂ = argmax_r  P(r | pt, path, position, voice, hw, p)

Here r̂ is the predicted role, and r ranges over the role labels plus "none". The term pt is the phrase type of the constituent, such as NP or PP. The path is the chain of tree nodes linking the constituent to the predicate. The term position records whether the constituent lies before or after the predicate. Voice marks active or passive. The term hw is the constituent's head word, and p is the predicate lemma. Because the full conditional is sparse, it was estimated by interpolating less specific sub-distributions arranged in a backoff lattice (Gildea & Jurafsky 2002; Gildea & Palmer 2002).

Labeling each node in isolation ignores the fact that the arguments of a verb form one structure. Core roles rarely repeat, and their order depends on voice. Joint models address this. One approach re-ranks n-best complete frames with a log-linear model (Toutanova et al. 2005; Toutanova et al. 2008). Another imposes hard constraints through ILP, such as no overlapping spans and no duplicated core labels (Punyakanok et al. 2008).

Neural systems often recast the whole problem as BIO sequence tagging relative to a marked predicate (Zhou & Xu 2015; He et al. 2017).

History and founding papers

Predicate–argument structure was used computationally before any treebank layer existed. Hindle grouped nouns semantically by the verbs they appeared with as parsed subjects and objects (Hindle 1990). The data foundation for SRL itself came from treebanks. The Penn Treebank added predicate–argument information to its syntactic trees (Marcus et al. 1994).

The Berkeley FrameNet project began annotating British National Corpus sentences with frame elements. At its first report it had nearly 10,000 sentences for about 200 lemmas (Baker et al. 1998). PropBank then layered verb-specific role labels over every verb occurrence in the Wall Street Journal Treebank (Kingsbury & Palmer 2002; Palmer et al. 2005).

Gildea and Jurafsky turned SRL into a statistical task. Their data were about 50,000 FrameNet-annotated sentences. They reached 82% role accuracy when constituent boundaries were given. When the system also had to find the arguments, it achieved 65% precision and 61% recall (Gildea & Jurafsky 2002). A companion study on PropBank quantified how much parsing matters. With gold trees, the system reached 71.1% precision and 64.4% recall. Automatic parses lowered this to 57.7/50.0, and flat chunks to 27.6/22.0 (Gildea & Palmer 2002).

The CoNLL shared tasks standardized evaluation. In CoNLL-2005, a predicted argument counted as correct only if both its span and its label were right. Training used 39,832 WSJ sentences. Nineteen teams took part, and the best system scored 79.44 F1 on the WSJ test. Every system lost roughly 10 points on the Brown out-of-domain set (Carreras & Màrquez 2005). The winning entry, later described in a journal article, combined outputs built on different parsers through ILP inference (Punyakanok et al. 2008). A later joint re-ranking model pushed WSJ test F1 to 80.32 (Toutanova et al. 2008). Tree kernels offered a way to automate feature engineering in the same setting (Moschitti et al. 2008).

CoNLL-2009 moved to dependency-based SRL in seven languages. A generic pipeline of logistic-regression classifiers with a global reranker averaged 80.31 labeled semantic F1 there, the second-best result (Björkelund et al. 2009). PropBank-style annotation was also extended to Chinese verbs and nominalizations (Xue 2008).

How it is used today

Neural models replaced hand-built features. A deep bidirectional LSTM with a CRF output layer, using no parse input, reached 81.07 F1 on CoNLL-2005 (Zhou & Xu 2015). An 8-layer highway BiLSTM ensemble raised this to 83.2 on CoNLL-2005 and 83.4 on CoNLL-2012 (He et al. 2017). Self-attention encoders achieved single-model scores of 83.4 and 82.7 on the same two benchmarks (Tan et al. 2017).

A parallel line of work brought syntax back in. One approach encodes dependency paths with LSTMs (Roth & Lapata 2016). Another runs graph convolutions over dependency trees. As an ensemble, it reached 89.1 labeled F1 on English CoNLL-2009 dependency-based SRL (Marcheggiani & Titov 2017). A third trains one attention head to follow dependency heads, as in the LISA model (Strubell et al. 2018).

Span-based models predict predicates and arguments jointly. With ELMo, such a model reached 86.0 F1 on the CoNLL-2005 WSJ test set with predicted rather than gold predicates (He et al. 2018). Simple BERT-based taggers used no syntactic features. They scored 88.8 span-based F1 on CoNLL-2005 and 92.4 dependency-based F1 on CoNLL-2009 (Shi & Lin 2019). The two families are scored differently, so their numbers are not directly comparable. On the FrameNet side, the SEMAFOR parser made frame-semantic analysis available off the shelf (Das et al. 2014).

Downstream uses include several areas:

Open questions

The role of syntax is still debated. With gold parses, syntax still promises gains of a few F1 points (He et al. 2017; Strubell et al. 2018). With automatically produced parses, the gap is smaller. In dependency-based SRL, syntax-aware systems now lead syntax-agnostic ones by only about 1–2 points (He et al. 2018).

Domain shift remains a persistent weakness. The roughly 10-point drop on Brown text seen at CoNLL-2005 has shrunk but not vanished (Carreras & Màrquez 2005; He et al. 2018).

Span boundaries, rather than role labels, cause most of the remaining errors (Tan et al. 2017). Independent span classifiers also sacrifice global consistency (He et al. 2018).

Annotation cost limits coverage and the spread to new languages. QA-SRL replaces fixed role inventories with templated wh-questions that non-experts can answer. Its guidelines run to 5 pages, compared with 89 for PropBank and 119 for FrameNet (He et al. 2015).

Finally, it is still unclear which inventory best serves downstream reasoning. The options range from verb-specific numbered arguments and frame-specific elements to broad thematic roles or natural-language questions.

Key papers

The works this page cites, the most cited first; citations as counted by OpenAlex plus the papers of this library that cite the work, read 2026-10-23.

More papers

Topics: 🎭 Semantic role labeling

Shelves: 🗣️ Linguistics & languages · 💬 Natural language processing

This page in Markdown → https://md.research.elamur.ai/free/en/concepts/semantic-role-labeling/