📄 Multilingual Semantic Role Labeling

Summary

The paper describes a semantic role labeling system built for the SRL-only closed track of the CoNLL-2009 shared task. The task covered seven languages: Catalan, Chinese, Czech, English, German, Japanese and Spanish. Predicates were already given, so the system only had to choose each predicate's sense, find its arguments and assign their labels. The authors built a chain of local L2-regularized logistic regression classifiers using LIBLINEAR. A beam search over these classifiers produces a small set of candidate propositions, which a global reranker then rescores using whole-proposition features. A greedy forward feature-selection procedure, run separately for each language, made multilingual adaptation largely automatic. It raised scores by about 10% over a fixed baseline feature set. The system used no lexicons and no hand-built linguistic resources. It averaged 80.31 labeled semantic F1, the second-best result overall. After the authors fixed a reranker training bug for Spanish, the average rose to 80.80. These results show that a simple, generic design can perform evenly across very different languages.

Key points

• The architecture has three parts:
  • Local models: per-lemma predicate disambiguation, binary argument identification (AI) and multiclass argument classification (AC).
  • Reranking: a global reranker in the style of Toutanova et al. (2005).
  • Combination: final scores calculated as P_Local^α × P_Reranker, with α = 1.
• Beam widths were k = l = 4, giving pools of 13 or 16 candidates. Reranker training data came from 5-fold jackknifing of the AI and AC models.
• Official results:
  • 80.31 average labeled semantic F1, ranking 2nd.
  • Best scores on Chinese and German, 2nd on English.
  • 74.38 on out-of-domain data, ranking 3rd.
• Spanish rose from 76.52 to 79.91 after the bug fix. The reranker added about 0.7 F1 on average but slowed parsing by a factor of 1.04 to 1.82.
• Chinese and German input parses had only 78.46 LAS, yet SRL F1 reached 78.60 and 79.71, suggesting the system tolerates noisy parses well.

Interesting ideas

• I1. Automatic per-language feature selection: The authors ran greedy forward selection over single features and then over feature pairs, instead of hand-tuning features for each language. This could transfer to any multilingual classification pipeline, including Ukrainian, where little engineering effort is available per language.
• I2. Composite labels as single classes: Where an argument had several dependency labels at once (Czech, Japanese), the labels were joined into one new class rather than handled with special machinery. This is a cheap way to handle multi-label edge cases in a standard multiclass setup.
• I3. Generate with local models, rescore globally: Local probabilistic classifiers propose a short list of candidates through beam search. A reranker then scores whole structures, and the two scores are combined log-linearly. This pattern suits many structured prediction problems where global constraints are hard to encode in local decisions.
• I4. Jackknifed reranker training: The reranker was trained on candidates produced by models that had not seen that part of the data. The gold answer was injected as a positive example whenever the beam missed it. This keeps the reranker's training data realistic rather than overly optimistic.
• I5. Splitting ambiguous categories, with a fallback: For English, nominal and verbal predicates got separate classifiers, plus a catch-all classifier for words the POS tagger mislabeled. This guards against errors propagating from upstream tools.