📄 A corpus of preposition supersenses in English web reviews

Summary

English prepositions are extremely frequent and highly polysemous, yet earlier annotated resources for their meaning cover only a few dozen common types, sample isolated tokens, or lack inter-annotator agreement figures. The authors build the first corpus where every preposition token in full documents carries a preposition supersense: a coarse, unlexicalized category such as DESTINATION, PURPOSE or TIME. These categories form an inheritance hierarchy modeled on VerbNet thematic roles. The data is the REVIEWS section of the English Web Treebank, which already had multiword-expression and noun/verb supersense layers. Tokens were double-annotated and then adjudicated. Annotator disagreements prompted revisions to the scheme, including removing the catch-all OTHER category. Two experts independently annotating the same sample agreed on about 88% of tokens. The authors take this as evidence that the corpus is dependable enough to be used both to train disambiguation systems and to benchmark them. The resource matters because it covers all preposition types and sits on top of existing lexical-semantic layers, which makes joint modeling of noun, verb and preposition supersenses possible.

Key points

• Source data: the EWT REVIEWS section, with 55,579 tokens, 3,812 sentences and 723 documents, using gold tokenization and PTB POS tags. Prepositions make up 8.5% of tokens.
• Coverage: 4,250 tokens carry supersenses, spanning 114 preposition types and 63 supersenses.
  • Nine prepositions account for 75% of tokens, while supersenses are spread far more evenly.
  • About half the tokens (2,100) are neither spatial (1,170) nor temporal (528).
• Special labels and MWEs: 582 tokens are MWEs (multiword prepositions or PP idioms). Another 588 get non-supersense labels:
  • 484 `i (bare infinitival to),
  • 83 `d (discourse expressions),
  • 21 plain ` (miscellaneous cases).
• Agreement:
  • Expert-to-expert adjudication reliability is 276/313 = 88%, with Cohen's κ = .878.
  • Per-preposition κ ranges from .59 (of) to 1.0 (to).
  • Agreement in the regular annotator batches was lower, mostly 60–78%.
• Standard split: the held-out test set has 447 sentences and 950 preposition instances (19.6%), leaving 3,888 for training. Sampling was stratified by supersense so that rare labels show up on both sides.

Interesting ideas

• I1. Label each function word with a small set of mnemonic, type-independent role categories rather than a per-word sense list, so annotators and models can handle open vocabulary without consulting a dictionary.
• I2. Organize roles in a hierarchy with multiple inheritance (DESTINATION inherits from both LOCATION and GOAL), so properties of a parent category automatically apply to its children.
• I3. Reserve explicit non-semantic labels for tokens doing purely syntactic or discourse work (bare infinitival to, discourse idioms, as…as comparatives) instead of forcing a semantic role onto them.
• I4. Estimate dataset quality by having the two adjudicators independently annotate the same sample and measuring their overlap, which approximates how much the labels would change under a different adjudicator.
• I5. Count a multiword PP idiom under the longest prefix that exists in the lexicon, so idioms can still be tallied per preposition type.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): preposition supersenses could tag the role of each PP attached to a VerbNet frame, so "for 30 years" becomes DURATION and "for vacations" becomes PURPOSE, giving an explainable reason for each role choice.
• I2 (world): the world builder could reason over parent categories, for example treating any LOCATION descendant as a place an entity can occupy, without a separate rule for every leaf role.
• I3 (frames): marking such tokens as structural prevents spurious roles in frames and keeps a visible record of why no role was assigned.
• I4 (learn): when several witness parsers or teacher tiers settle silver data, an overlap sample judged independently by two teachers could give a measured reliability ceiling for that silver set.
• I5 (tokens): the same longest-known-prefix rule could link MWE lemmas back to their head preposition entry, so lexical lookups and statistics still work for idiomatic phrases.