📄 Building a Large-Scale Annotated Chinese Corpus

Summary

The paper reports on building the Penn Chinese Treebank (CTB) and explains how the team handled three practical problems: making annotation faster, keeping it accurate and consistent, and making the corpus useful. The first release, CTB-I, has 100,000 words of Xinhua newswire. It is segmented, POS-tagged and syntactically bracketed. The authors used it to train a maximum entropy segmenter, a retrained Ratnaparkhi tagger and Chiang's statistical parser. These tools then pre-annotate text for the larger CTB-II, and human annotators correct the output. Each new batch of annotated data allows better tools to be trained, so the process feeds itself. The guidelines also deliberately avoid elaborate theory-driven structures when a simpler tree loses no information. Quality is maintained by checking annotators' agreement and accuracy and by running automatic consistency checkers. The authors argue that this workflow applies to corpus building in any language. They also identify what the syntactic layer cannot yet support, namely predicate-argument and coreference information.

Key points

• CTB-I contains 100,000 words of Xinhua newswire from 1994–1998 and is distributed through LDC. CTB-II targets 400,000 words and was planned for early 2003. It adds People's Daily, Hong Kong newswire and texts translated into Chinese.
• The segmenter treats segmentation as character tagging with four labels: LL, MM, RR and LR (single-character word). Trained on 80k words and tested on 20k, it reaches about 91% F-measure.
• The Ratnaparkhi maximum entropy tagger, retrained on the same split, reaches about 93% accuracy. Correcting segmenter and tagger output takes roughly half the time of annotating from scratch.
• Chiang's parser, trained on 80k words and tested on 10k, scores 73.9% labeled precision and 72.2% labeled recall. Its value as a preprocessor was tested in a controlled experiment on a randomly chosen 13,469-word chunk. The chunk was split into two halves: one annotated from scratch, the other parser-preprocessed and then corrected.
• The bracketing scheme encodes three core relations: complementation, adjunction and coordination. Functional tags (-SBJ, -TPC, -OBJ, -EXT, -TMP, -ADV) supply additional information. LexTract and a corpus search tool catch mechanical errors such as missing brackets or invalid labels.

Interesting ideas

• I1. Bootstrapping annotation with your own tools: Train preprocessors on a small first release, use them to pre-annotate the next batch, and retrain as data grows. This turns annotation into correction work and suits low-resource language projects starting from a seed corpus.
• I2. Test whether preprocessing actually helps: Instead of assuming that pre-parsing saves time despite error backtracking, the authors blindly split a sample and compared annotation from scratch against correction. Running this kind of A/B test is a cheap way to justify, or reject, model-assisted annotation in any pipeline.
• I3. Simplify trees only when nothing is lost: Raising verbs take a bare VP complement instead of a trace-filled clause. This works because the original structure can be restored automatically. The design principle is to keep guidelines simple wherever the full analysis can be recovered deterministically.
• I4. Leave hard ambiguity flat on purpose: The internal structure of noun-noun compounds is not bracketed. That trade-off spares annotators slow, inconsistent decisions while keeping the major grammatical relations intact.
• I5. Segmentation as character position tagging: Labeling each character as left, middle, right or standalone turns word segmentation into a standard sequence-labeling problem. Training data follows directly from any segmented corpus, and the same framing works for other tokenization or chunking tasks.