📄 SERRANT: a syntactic classifier for English Grammatical Error Types

Summary

Grammatical error correction work often sorts edits (an erroneous span plus its fix) into types, for analysis and evaluation. English has two automatic edit classifiers with different taxonomies. ERRANT is rule-based and easy to read, and it served as the official taxonomy of the BEA shared task. However, some of its classes are vague or inconsistent, notably OTHER and MORPH. SErCl is cross-lingual and builds a type by pairing the UD features (UPOS, dependency label, morphological features) of the span's head before and after correction, as in NOUN→VERB. SERRANT merges the two. It returns ERRANT's label by default and switches to SErCl's syntactic label, or a refined label, wherever ERRANT is uninformative. The paper gives a hand-designed set of override rules plus suffixes for word choice and multi-word edits. It illustrates them with invented sentences and with real level A learner sentences from the W&I corpus. The paper reports no quantitative evaluation. The value is a single, more informative English error taxonomy that runs on any corpus with corrections. Code is at github.com/matanel-oren/serrant.

Key points

• SERRANT keeps ERRANT's R/M/U prefixes (replacement, missing, unnecessary) and its m2 output format. A deleted verb, for example, becomes U:VERB rather than SErCl's VERB→NONE.
• Eight special-case rules override ERRANT:
  • OTHER is replaced by SErCl types.
  • MORPH is replaced by SErCl types.
  • Capitalisation into a proper noun that is not sentence-initial becomes X→PROPN.
  • VERB is split into AUX where needed.
  • Noun-to-verb edits become NOUN→VERB instead of VERB:FORM.
  • PRON↔DET swaps get their own types.
  • VERB:TENSE is kept only for be/have/will; edits between modals get a Modal suffix, and the rest go to SErCl.
• Edits involving Intj, Num, Sym, X or Punct stay as OTHER because they are considered unreliable. PROPN is also distrusted, since the parser assigns it to misspelled words as a fallback; PROPN→PROPN is still kept.
• Two suffixes are added:
  • "WC" (word choice) marks a mismatch in lemmas while the POS is unchanged, e.g. consume→eat. eat→ate would not get it.
  • "MW" marks multi-word edits that do not already fall under a named type.
• The design goal is informativeness. ERRANT's subcategories are preserved so users can merge or ignore classes as they need.

Interesting ideas

• I1. Use a human-readable rule classifier by default and fall back to a structural, feature-pair description only where the readable one gives up, so every label stays interpretable.
• I2. Describe any change as "features of the head before → features after" (UPOS, deprel, morph), which works across languages and needs no hand-made taxonomy.
• I3. Mark categories unreliable when you know the upstream tool uses them as a fallback (here PROPN for misspellings), but trust them when the same label appears on both sides.
• I4. Separate a lemma change from a pure inflection change (the WC suffix), distinguishing "wrong word" from "wrong form" within one POS.
• I5. Use position context to refine a shallow category: a capitalisation fix that is not sentence-initial and yields a proper noun is treated as a meaning change, not mere spelling.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): self-induced repair rules could label each disagreement between witness parsers with a readable category first, and a raw UD feature pair only when no category fits.
• I2 (tree): diffs produced by UD dialect conversion or by comparing gparse against the perceptron parser could be summarised as such label pairs, showing which transformations dominate.
• I3 (tokens): the tagger could flag PROPN on unknown, lowercase or near-dictionary words as suspect before such tokens become entities.
• I4 (back): when checking the text → graph → text compressor, round-trip differences could be split into lemma substitutions (meaning drift) versus inflection slips (surface generation bugs).
• I5 (world): entity creation could use the same cue, so "apple" vs "Apple" mid-sentence decides between a common object and a named organisation in the world of the text.