📄 An Annotation Scheme for Free Word Order Languages

Summary

Treebank formats built for English rely on a context-free phrase-structure skeleton, with traces linking moved material. For free word order languages such as German, this design struggles. Discontinuous constituents are frequent, so traces multiply, and the resulting trees drift away from predicate-argument structure. The authors propose a different scheme that puts argument structure at the center. It uses flat, unordered trees whose branches may cross, and it has no empty categories. Grammatical functions such as subject, modifier and head are stored as edge labels, while syntactic categories are node labels and POS tags. When one element serves several roles, as with equi verbs, extra secondary edges are added. The authors also built an annotation tool that suggests function labels with a statistical tagger and asks the annotator to check uncertain choices. About 1100 German sentences have been annotated so far. The tagger reaches 95% overall accuracy, and annotation time per sentence drops by about 25%. The work addresses the lack of syntactically annotated German corpora and yields data that is closer to semantics and less tied to any one syntactic theory.

Key points

• The scheme has three strata:
  • argument structure as unordered trees with crossing branches;
  • grammatical functions on edges (e.g. SB, MO, HD, OC, OA, DA, GL, GR, PM, SVP, NK);
  • categories on nodes plus POS tags on words.
• The tagset is small: 16 node labels and 34 function tags. There are no bar levels or intermediate projections, and no empty categories.
• Function labels are assigned by a trigram tagger, evaluated with 10 random 90/10 train/test splits.
  • 90% of assignments are judged reliable and applied automatically, with 97% accuracy.
  • The remaining 10% are flagged for confirmation and are correct 75% of the time.
  • Overall accuracy is 95%. Most errors involve telling subjects apart from the various object types.
• Partial automation cuts average annotation time from roughly 4 to roughly 3 minutes per sentence.
• A planned follow-up in the Verbmobil project targets 30,000 spoken German sentences, plus a similar amount of English and Japanese.

Interesting ideas

• I1. Headedness as a label, not a structure. Marking the head with an HD edge label lets headless constructions (e.g. copula-less predicatives) be annotated without inventing empty heads. This is useful for any scheme that must handle "deficient" constructions.
• I2. Flat trees with rich labels. Keeping the structure shallow and pushing detail into function labels reduces disagreement between annotators over attachment levels. The same trade-off applies when designing UD-style or custom treebanks.
• I3. Flagging uncertain predictions. The tool compares the best label's probability with its strongest competitor and asks the human only when the ratio is close. This cheap uncertainty check fits pre-annotation pipelines for many tasks.
• I4. Two-stage annotation. A first pass covers structure and coarse functions. Later passes add finer details such as thematic roles, scope and anaphora, one level at a time. This keeps early annotation fast and consistent.
• I5. Annotating the common minimum. Where theories disagree, such as NP heads or the DP analysis, only the analysis shared across them is encoded. Theory-specific representations can then be derived later, which makes the resource reusable by different communities.