📄 Automatic Identification of English Collocation Errors Based on Dependency Relations

Summary

English learners often combine words in ways native speakers would not, and finding these miscollocations by hand in learner corpora is slow. Gao builds a checker that flags a word pair as suspicious when its grammatical link never appears in native text. A 14.5-million-word reference corpus of authentic English is parsed with the Stanford parser. Lemma-level head–collocate pairs for six relation types are stored in a database: subject-verb, verb-object, verb-adverb, noun-noun, adjective-noun and adverb-adjective. Learner sentences are spell-corrected and parsed the same way, and each relation is looked up in the database. Any pair that is not found becomes an error candidate. On the error-tagged st2 portion of CLEC, the system reaches about 0.75 precision overall, but recall is considerably lower. The author argues the tool can help learners correct themselves, reduce teachers' marking load, and let SLA researchers collect miscollocation patterns automatically. Examples of such patterns are confusion among support verbs and among near synonyms.

Key points

• Data: the reference corpus has 14.5M words of authentic English. The test set is CLEC subcorpus st2 (251,558 tokens), whose collocation error tags CC1–CC6 allow automatic scoring.

• Precision by error type:

  Type	Relation	Precision
  CC1	noun-noun	0.77
  CC2	noun-verb	0.87 (best)
  CC3	verb-noun	0.72
  CC4	adjective-noun	0.75
  CC5	verb-adverb	0.83
  CC6	adverb-adjective	0.63 (worst)

  Overall precision is about 0.75. Recall is not given as a number, only described as much lower.

• Pipeline resources:

  • Stanford parser typed dependencies.
  • WordNet 3.1 for lemmatization.
  • Aspell plus a Google 1T n-gram language model for spelling correction, combining minimal edit distance with n-gram probability.

• Database design: each record stores headword, POS, relation, collocate and collocate POS. Subject-position and object-position nouns are kept apart, and prepositions are included for patterns like "rely on". Pairs involving personal pronouns or proper names are skipped using POS tags.

• Failure sources:

  • parser mistakes, especially with ellipsis, complex sentences and wrong NP heads;
  • collocations that fall outside the stored relation types;
  • valid pairs missing from the small reference corpus;
  • polysemy and constructions, e.g. "make stomach" and "take university" are fine in certain contexts.

Interesting ideas

• I1. Treating a lemma-level dependency triple that is unseen in a reference database as an anomaly signal is a simple, explainable detector that needs no trained classifier.
• I2. Excluding pronouns and proper names before the lookup avoids false alarms from open-ended, sparse slots whose combinations a finite corpus can never cover.
• I3. Correcting spelling first, by combining edit distance with an n-gram probability, recovers relations that would otherwise be lost to typos.
• I4. Keying the collocation table on syntactic role (subject noun vs. object noun) rather than bare POS sharpens the lookup and enables role-specific queries such as "verbs that take responsibility as object".
• I5. The paper notes that correct word pairs do not guarantee a correct phrase, and that constructions like "take X for instance" break pairwise checks; plausibility must be judged at a larger unit.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): silver trees containing head–relation–dependent triples never attested in a gold treebank could be routed to extra witness parsers or the LLM teacher for review.
• I2 (frames): when checking selectional plausibility of semantic roles, named entities and pronouns should be backed off to their entity type before consulting verb–argument statistics.
• I3 (tokens): a normalization pass ranking candidates by edit distance and context probability before tagging would give the parser cleaner lemmas on noisy student text.
• I4 (world): compiled verb knowledge could store argument preferences per VerbNet role slot, so world-building can judge whether an entity fits as agent or as patient.
• I5 (frames): acceptability checks should run on whole frames with construction identity, not on isolated edges, so idiomatic or construction-licensed pairs are not wrongly rejected.