📄 Czech Grammar Error Correction with a Large and Diverse Corpus

Summary

Data for grammatical error correction (GEC) is scarce outside English, and even English corpora mostly contain texts by language learners. The authors address both gaps for Czech with GECCC (Grammar Error Correction Corpus for Czech). It has 83,058 sentences across four domains: essays by native school pupils, informal web discussions, essays by Romani-ethnolect children and teenagers, and texts by non-native learners. The material comes from the AKCES resources, MERLIN and AKCES-GEC, plus newly annotated posts from the Czech Facebook Dataset and novinky.cz. The sources used different annotation schemes, so the development and test sets, and part of the training data, were re-annotated under a single scheme. The splits are balanced across domains and stay compatible with AKCES-GEC. The authors benchmark several Czech GEC systems, including Transformer-based ones, and check common metrics against human judgments in each domain. They recommend M2 with β = 0.5. The resource gives a strong baseline and a much broader test bed for non-English GEC.

Key points

• GECCC has 83,058 sentences, compared with 47,371 in AKCES-GEC. The authors call it the largest non-English GEC corpus. It is released under CC BY-SA 4.0 in two formats: tokenized M2 (using the UDPipe tokenizer) and detokenized text aligned at sentence, paragraph and document level.
• The overall token error rate is 18.2%, much higher than the 8.2% of the CoNLL-2014 test set. Error density varies widely by domain, from dense non-native essays to cleaner web texts.
• In the meta-evaluation, M2 0.5 and ERRANT 0.5 correlate best with human judgments, both with a system-level Pearson r of 98.28. M2 0.5 is preferred because it is language-agnostic, whereas ERRANT needs a POS tagger, lemmatizer and language-specific rules. The optimal β lies between 0.4 and 0.5.
• The spell-checker Korektor makes harmful edits in the Native Formal and Native Web Informal domains, often by changing named entities. M2 cannot reveal this because it does not allow scores below the unchanged-input baseline.
• Neural models fine-tuned on all domains improve results in every domain. However, a large gap to the human reference remains, and the Second Learners domain offers the most room for improvement.