📄 Machine Learning Approaches for Amharic Parts-of-speech Tagging

Summary

The authors aim to push Amharic part-of-speech tagging past roughly 91% accuracy, which earlier taggers had not exceeded. Amharic is a Semitic language with rich root-and-pattern morphology. They started from the ELRC news corpus, which has 31 tags. Because no cleaned release of it was available, they re-cleaned it and then built two richer resources:

• ELRC-Extended: the tag set refined to 51 tags.
• ELRCQB: 62 tags, adding manually tagged Amharic Quran and Bible texts.

They compared three taggers, Brill, TnT and a linear-chain CRF (sklearn-crfsuite, L-BFGS). The CRF uses hand-built orthographic and affix features, with its regularisation tuned by grid search. All results come from 10-fold cross-validation, reported separately for known and unknown words.

The CRF was the best tagger on every dataset, reaching 95.87% on ELRC-Extended. Its main advantage was on words unseen in training. Adding the religious texts lowered accuracy, which the authors attribute to domain and style differences. The work matters because it supplies larger tagged data and a stronger tagger for an under-resourced, morphologically complex language.

Key points

• Data: the ELRC corpus has about 210,000 words, with 11 basic tags refined to 31.
  • ELRC-Extended (51 tags) adds plural variants, such as NS for plural nouns, and merged preposition/conjunction-with-adverb tags.
  • ELRCQB has 62 tags and 33,940 sentences (440,941 words).
• CRF features:
  • sentence-initial/final position and hyphen;
  • digit and alphanumeric flags for the current, previous and next word;
  • prefixes and suffixes of length 1–3;
  • the previous one and two tags.
• Tuning: grid search over sklearn-crfsuite chose c1 = 0.064 and c2 = 0.002.
• CRFSuit average accuracy: 86.44% on ELRC, 95.87% on ELRC-Extended and about 92.2% on ELRCQB.
  • Brill and TnT scored about 81.6–82.3%, 93.9% and 88.5–88.7% on the same three datasets.
  • The earlier best result was 90.95% (CRF, Gebre 2010).
• Unknown words on ELRC: the CRF reached 75.79%, versus about 26% for Brill and TnT, which used n-gram backoff to the most frequent tag. On known words, Brill and TnT were slightly ahead of the CRF.

Interesting ideas

• I1. Each fold's accuracy is reported separately for known and unknown words, together with how many unknown tokens that fold contains, which shows where a tagger actually wins (here, almost entirely on unseen words).
• I2. Cheap prefix and suffix features (lengths 1–3) let a discriminative tagger guess tags for unseen words far better than n-gram backoff, even though n-gram methods do slightly better on familiar words.
• I3. Building morphological number into the tag itself (for example, plural noun as a separate tag) gives downstream tasks more information and, in this paper, coincided with higher tagging accuracy.
• I4. Adding out-of-domain text (religious scripture) increased the unknown-word rate and lowered accuracy, which shows that a merged evaluation can hide domain shift.
• I5. The confusion matrix is shown for only the 20 most frequent tags, which keeps a 60-class error analysis readable.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): Report the en tagger's accuracy split into in-vocabulary and OOV tokens per test set, so gains from new features are attributed correctly.
• I2 (tokens): Make sure the en tagger and lemmatizer use affix features for OOV words like rare derivations and technical terms, and keep lexicon lookup for known words.
• I3 (tokens): Check whether the en morphology outputs Number and similar features as explicit, verifiable fields that frames and world can use when tracking entity counts.
• I4 (learn): Evaluate silver-trained models separately per genre (school texts, fables, word problems) instead of on one pooled set, so style-driven regressions become visible.
• I5 (learn): Add a top-N confusion report for POS and dependency labels to evaluation runs, so the most frequent error pairs can feed repair rules.