📄 Towards a Semantic Annotation of English Television News - Building and Evaluating a Constraint Grammar FrameNet

Summary

The paper tackles semantic annotation of a very large multimodal monitor corpus, the UCLA Communications Studies Archive. Its time-aligned transcripts of English TV news are far too big to mark up by hand. The author builds a rule-based layer for verb senses and semantic roles. It sits on top of the EngGram Constraint Grammar parser, which already supplies morphosyntax and dependencies. The lexical resource is the EngGram FrameNet (EFN), derived largely by porting material from the Danish DanGram framenet. In EFN, every verb sense is tied to a syntactic valency frame. Each argument slot records a thematic role, a syntactic function, a morphosyntactic form and typical semantic prototypes. A converter compiles each frame into a CG mapping rule that checks dependency-linked arguments, so no annotated training corpus is needed. On a manually inspected sample of news transcripts, verb-sense tagging reaches roughly 80% F-score, about 5 points below the Danish system. Most errors come from senses missing from the lexicon and from auxiliary/main-verb confusions in spoken language. The approach shows that lexicon-driven, transparent semantic tagging is feasible for noisy spoken data.

Key points

• Corpus: UCLA CSA holds about 150,000 hours of recorded news and grows by about 100 programs per day. Annotation targets the second-by-second-aligned transcript layer.
• EFN size and inventory:
  • 7820 verb senses for 4774 verb types, with 10,800 valency frames.
  • About 500 frame types, inspired by VerbNet classes.
  • About 35 core thematic roles plus 10–15 adverbial roles assigned from syntactic context.
  • Roughly 2/3 of frames carry a best-guess link to a Berkeley FrameNet sense.
• Role distribution is skewed: the top 5 roles cover over half of all role tags, with §TH alone at 21.91%. Roles correlate clearly with functions; for example, @DAT is mostly §BEN (52.8%) or §REC (41.9%).
• Evaluation on TV news: 495 of 615 frames were correct, giving recall 80.49%, precision 79.32% and F-score 79.91. Ignoring POS/auxiliary errors raises the F-score to 80.75. The Danish counterpart scores about 85.
• Error analysis:
  • About 40% of false positives involved a gold sense absent from EFN.
  • Verbs with tagging errors averaged 10.26 potential senses, compared with 4.7 for all verbs in the sample.
  • Support-verb constructions are encoded with an empty §INC role, and about 14.6% of valency entries contain incorporated material.

Interesting ideas

• I1. Design the sense inventory so that no two senses of a verb share an identical argument profile, meaning each pair is separated by some syntactic or semantic property of an argument, which makes every sense decidable from parser output alone.
• I2. Keep knowledge in a lexicon and mechanically compile each entry into a dependency-checking rule, rather than hand-annotating a training corpus.
• I3. Handle support-verb and particle constructions (take care of, take place) by anchoring all dependencies and the frame label on the verb, and giving the noun an empty "incorporated" role.
• I4. Let a lexicographer-controlled ranking choose a default frame when context slots are missing, and let lower-ranked frames fire only when all their conditions are met.
• I5. Break down errors by whether the gold sense even exists in the lexicon, separating coverage gaps from disambiguation failures.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): VerbNet-based frame entries in prag could be audited for "indistinguishable" sense pairs, which can then be merged or given an extra slot constraint so selection stays explainable.
• I2 (world): verb-class knowledge could be authored once as data and compiled into frame-matching rules, so fixing one lexicon entry immediately fixes all its uses.
• I3 (frames): light-verb constructions could be resolved consistently on the verbal head, with the incorporated noun marked as non-argument so role assignment stays uniform.
• I4 (frames): a transparent fallback ordering for frame selection would keep decisions explainable when the tree lacks arguments.
• I5 (learn): evaluation reports could tag each frame error as "missing entry" versus "wrong choice", pointing work toward lexicon growth or toward rule repair.