📄 Analysis of English Spelling Errors in a Word-Typing Game

Summary

The paper asks whether a word-typing game can serve as a cheap and reliable source of English spelling errors. Earlier methods either mined errors from tweets by edit distance, where the intended word is uncertain, or paid crowd workers to type while their keystrokes were logged. The authors built two single-player games using 842 Basic English words. One is a standard game that judges the word on Enter. The other is a correctable game that also logs backspace edits. This lets them separate errors players fixed themselves (corrected) from errors that slipped through (uncorrected). Because the target word is always known, the error and the intended word are aligned exactly, using dynamic programming. The authors compared the logged errors with the keystroke dataset of Baba and Suzuki (2012). The edit-distance profiles, error-type proportions and the corrected-to-uncorrected ratio closely matched that dataset. Error-frequency rankings also correlated strongly across the games and with the reference. The conclusion is that game logs are a valid, low-cost substitute for crowdsourced error collection.

Key points

• Common game: 7 computer-science students typed 4,724 words, yielding 712 uncorrected errors.
• Correctable game: 19 students typed 21,468 words over three days, yielding 3,883 corrected and 1,334 uncorrected errors.
• Edit distance: nearly all errors fall within distance 1.
  • Corrected errors: 3,566 at distance 1, 148 at distance 2, 7 at distance 3.
  • Uncorrected errors: 1,099 at distance 1, 95 at distance 2, 15 at distance 3.
• Ratio: corrected to uncorrected errors was 3.00, against 2.94 in Baba and Suzuki (2012).
• Correlation: Pearson correlation of frequency-ranked errors was 0.918 between the two games and 0.886 between the correctable game and the Baba and Suzuki data.
• Error types: substitution dominates corrected errors. Uncorrected errors are mostly substitutions and deletions, since dropped letters tend to go unnoticed.
• Design details: each game lasts 50 words. A correct word earns +100 points and a wrong one costs −10, with a speed bonus and a visible high score. Translations are shown on screen for second-language learners.

Interesting ideas

• I1. When the intended form is known in advance, every noisy input pairs exactly with its target, so no edit-distance guessing is needed; no cross-checking between players is needed either, because the dictionary is the referee.
• I2. Logging backspaces splits errors into ones people catch and ones they miss, and the two have different type distributions (deletions escape notice).
• I3. A cheap data source is validated by comparing its statistics with an established dataset: edit-distance histogram, type proportions, a key ratio, and the correlation of frequency-ranked items.
• I4. Only the long-distance tail (distance ≥3) was inspected by hand, and it was sorted into categories such as genuine errors, accidental keypresses, give-ups and junk.
• I5. Letter-level confusion matrices for substitutions capture keyboard-adjacency and vowel-confusion patterns.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): we could build a typo-robustness test set for the en crate by having users or LLMs retype known sentences, giving gold pairs of noisy and clean tokens.
• I2 (learn): noise-injection negative controls should sample from the uncorrected profile, which is heavy on substitutions and deletions, since that is the noise that actually reaches finished text.
• I3 (learn): silver data agreed by witness parsers could be checked the same way, by correlating its label or relation frequency profile against gold treebank statistics before it is trusted.
• I4 (learn): self-induced repair rules could be reviewed by triaging only the outlier repairs into explicit categories, keeping manual effort small.
• I5 (tokens): such a matrix could weight candidate generation for unknown words in the tokenizer or lemmatizer, so that each suggested repair is explainable by a likely key slip.