📄 WikiWoods: Syntacto-Semantic Annotation for English Wikipedia

Summary

WikiWoods tackles the lack of deep linguistic annotation for English Wikipedia, a huge and fairly clean text collection that NLP researchers widely use. The authors built a pipeline with four main steps. It first extracts readable text from wiki markup, aiming to keep what a reader actually sees and drop tables, infoboxes and metadata. It then splits documents into sentence-like units and parses them with the HPSG-based LinGO English Resource Grammar (ERG), using a statistical parse ranker trained on the WeScience Treebank. Finally, it exports derivation trees, conventional constituent trees and Minimal Recursion Semantics (MRS) logical forms. A hand-corrected gold subset on NLP-related articles uses the same format. In a preliminary run on about half a million utterances, grammar coverage was just under 85%. A manual check of 1,000 top-ranked parses found over 83% correct or nearly correct. This matters because the resource offers semantics at Wikipedia scale, normalized across diathesis alternations, for tasks such as lexical semantic acquisition and ontology learning.

Key points

• The annotation comes from the ERG (HPSG) and includes three layers:
  • derivation trees labeled with construction and lexical-entry identifiers;
  • full feature structures averaging about 250 attribute–value pairs;
  • customizable constituent trees with 78 default category labels;
  • unresolved MRS with event, instance and handle variables.
• The manual evaluation covered 1,000 sentences:
  • 657 were judged correct, 181 nearly correct (one or two minor errors) and 147 incorrect.
  • Error rates rise with sentence length.
• About 15% of utterances get no parse. The first release (May 2010) still yields roughly 47 million annotated utterances.
• Parsing all of Wikipedia was estimated at about 100,000 CPU hours, or around ten days on a 6000-core HPC cluster at the University of Oslo.
• Preprocessing works on raw markup with cascades of regular expressions rather than a full wiki-markup parser.
  • Everything after sections such as See Also, References or External Links is dropped.
  • Templates that mark language-specific content, such as foreign words or IPA, are kept and handed to a custom tokenizer.

Interesting ideas

• I1. The MRS layer gives passive/active and dative-shift variants the same semantic roles, while leaving unmarked word senses underspecified rather than guessing.
• I2. The tree labels are not fixed: templates map rich internal analyses to category symbols, so users can re-derive custom trees, for example with flattened VPs or without unary nodes, from stored derivations.
• I3. Typographic markup like italics is treated the same way as punctuation (pseudo-affixes), so italic spans are bracketed as quoted chunks and then promoted to name-like phrases.
• I4. The manual evaluation uses a three-way grade (correct / nearly correct / incorrect), defining "nearly" as at most two minor errors that do not change meaning, and reports results by sentence length.
• I5. Text extraction aims to reproduce what a human reader is shown, and cuts boilerplate sections, along with anything after them, as the end of the article body.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): Frames could be normalized across voice and dative alternations before role assignment, with sense left as an explicit open slot when syntax gives no evidence.
• I2 (tree): The richest analysis could be stored once, with UD dialects and other outputs generated by editable mapping rules instead of being re-parsed or hand-converted.
• I3 (tokens): The tokenizer could carry markup spans (italics, links, code) as bracketing hints, so foreign phrases and use–mention cases become single name-like units.
• I4 (learn): Silver-data audits could adopt this graded, length-bucketed scheme to separate meaning-preserving slips like PP attachment from damaging errors.
• I5 (world): When building a text's world from web pages, the same rendered-view principle could keep tables and reference lists from injecting spurious entities.