📄 Comlex Syntax: Building a Computational Lexicon

Summary

The authors describe COMLEX Syntax, an English computational lexicon built at NYU and distributed through the Linguistic Data Consortium for research and commercial use by its members. It covers about 38,000 headwords drawn from a word list derived from the Oxford Advanced Learner's Dictionary (OALD). It aims to record syntactic properties, especially complement structure, in more detail and in a more theory-neutral way than OALD or LDOCE. Entries are nested typed feature-value lists. Complement types are formally defined frames that pair constituent structure with grammatical functions and control or raising information. Four linguistics graduate students ("elves") entered the entries by hand, using a menu-driven tool with an attached corpus concordance. An initial dictionary of nouns, verbs and adjectives was completed in May 1994. The paper's main contribution is an analysis of lexicographic error, sorted into four kinds: misclassification, omitted features, excess features and fuzzy features. The authors measure inter-annotator agreement and recall against a tagged corpus of "j" verbs. Their key finding is that having two annotators code each verb substantially improves coverage.

Key points

• The feature inventory has 92 subcategorization features for verbs, 14 for adjectives and 9 for nouns.
  • It distinguishes four control types: subject, object, variable and arbitrary.
  • Control can vary by complement or even by preposition, as with "blame … on" versus "blame … for".
• Each complement frame specifies four parts:
  • constituent structure (:cs);
  • grammatical structure (:gs);
  • features;
  • an example.
• Annotators agreed on argument vs. adjunct decisions 93–94% of the time, rising to 96–98% on examples not flagged as difficult or figurative.
• Recall was measured against 150 "j" verbs tagged in newspaper text outside the concordance:
  • Complements only: individual elves reached 82–96% (average 90%), the four-elf union 100%, and pairs of elves about 97% on average.
  • Including prepositions/particles: the average was 76% without the directional-preposition option and 84% with it.
  • LDOCE comparison: LDOCE covered 73% of tagged complements, or 79% under a softer comparison.
  • Spurious features: these made up only 2–6% of entries.
• The concordance corpus grew from the San Jose Mercury News to include:
  • the Brown corpus;
  • Library of America literary texts;
  • DOE scientific abstracts;
  • the Wall Street Journal.
• The authors planned to tag at least 25 instances per verb in Brown, aligned with WordNet sense tagging.

Interesting ideas

• I1. Dual encoding of control. Control and raising are stored both structurally, through indexed links between constituent and functional roles, and as an explicit feature. This redundancy lets different downstream users pick whichever form suits their parser.
• I2. Treating error as a measurable category. Errors are split into classification, omission, overgeneration and gradience. Each category gets its own countermeasure and measurement, which is a useful checklist for any hand-built linguistic resource.
• I3. Recall against held-out text. Annotators' lexicon entries are scored against complements tagged in text they never saw. This gives a practical recall estimate, and the authors also check whether missed items would actually break parsing or simply fall back to adjunct readings.
• I4. Macro option for open classes. A single P-DIR switch fills in the whole set of directional prepositions for motion verbs. It accepts slight overgeneration to avoid systematic omissions, a useful trade-off wherever annotators must enumerate long, predictable lists.
• I5. Targeted double annotation. Two annotators code only the critical, high-frequency entries rather than everything. This captures most of the union's coverage gain at a fraction of the cost, and a lexicon-linked tagged corpus can supply frequency information for gradient features.