Elamur Research⌕
☰
home / papers / 2015 / CUFE@QALB-2015 Shared Task: Arabic Error Correction System

CUFE@QALB-2015 Shared Task: Arabic Error Correction System

Paper · 2015 · Michael Nawar

Summary

This paper describes a Cairo University entry to the QALB-2015 shared task on automatic correction of Arabic text. The task had two tracks: comments by native speakers on Aljazeera and texts written by learners of Arabic (L2). The authors extend their earlier probabilistic rule-based corrector (Nawar and Ragheb, 2014). Correction rules are mined from the training data. Each rule's reliability is the share of its edits on the training set that turn out correct, and rules are ranked by that score. At test time the rules are applied to the whole test set one by one, best first. The system stops when an estimated F-score, computed without gold labels, stops rising.

There are two additions. First, word patterns (root-and-template shapes) come from a modified BAMA-v2.0 analyzer and support more general rules. Second, a CRF-based grammatical tagger replaces the older handcrafted analyzer and drives syntactic rules such as case endings. Measured with the M2 scorer, the system reaches F = 0.7287 on Alj-test-2015 and 0.3569 on L2-test-2015. The authors also point to weaknesses of F-score as a metric for this task.

Key points

  • Each rule gets a probability: the correct edits it makes on training data divided by all the edits it performs. When rules share a trigger word, only the most reliable one is kept.
  • The number of gold edits in the test set is estimated from the training error rate per word. This lets the system compute expected precision, recall and F after each rule and stop when F drops.
  • BAMA-v2.0 was modified to give each stem a pattern, and the word's pattern is built from the stem pattern plus its affixes. Loanwords and fixed function words get no pattern.
  • Pattern rules target:
    • a missing or extra definite article "Al" (frequent in L2 text),
    • gender and number mismatches,
    • simple punctuation.
  • Syntactic rules rely on CRF suite tags, using a −3/+3 token window with these features: POS, chunk, root, pattern, definiteness, gender and number. Example rule: genitive plural or dual nouns ending in "wn" or "An" are changed to end in "yn".
  • M2 results:
    • Alj-dev-2014: 0.6872
    • Alj-test-2014: 0.6668
    • Alj-test-2015: 0.7287
    • L2-dev-2015: 0.5673
    • L2-test-2015: 0.3569

Interesting ideas

  • I1. The system can judge on unlabeled data whether to keep applying rules, by combining each rule's training precision with an expected count of errors.
  • I2. Because the goal is global F, a rule with precision below 0.5 can still be worth applying, and a strong rule may be cut off partway through.
  • I3. Lexical rules generalize better when written over abstract word shapes (stem template plus affixes), with words that do not follow regular derivation left out.
  • I4. Case and number errors are fixed by first tagging grammatical function, then applying agreement constraints (e.g., after a preposition, or for an adjective or conjunct of a genitive noun).
  • I5. A whole-token metric gives no credit when only one of two errors in a word is fixed, so the authors argue for a finer-grained metric.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): self-induced repair rules could be ordered by their precision on silver data and applied until the estimated gain turns negative, which gives an explainable stopping point.
  • I2 (learn): when tuning repair rules, record which metric drives each accept/reject decision, so that negative controls can catch cases where gaming the metric hurts per-item correctness.
  • I3 (tokens): normalization or spelling-repair rules in crate en could be keyed on morphological shape classes rather than surface strings, with irregular and closed-class words excluded.
  • I4 (tree): agreement checks along UD relations such as nsubj, amod and conj could flag likely input errors or parse mistakes in a traceable way.
  • I5 (learn): evaluate repairs per feature (case, number, spelling) rather than per token, so that partial progress is visible.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksEducation · 203 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Michael Nawar. 2015. CUFE@QALB-2015 Shared Task: Arabic Error Correction System. Proceedings of the Second Workshop on Arabic Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w15-3215

@inproceedings{nawar2015cufe,
  title = {{CUFE@QALB-2015 Shared Task: Arabic Error Correction System}},
  author = {Michael Nawar},
  booktitle = {Proceedings of the Second Workshop on Arabic Natural Language Processing},
  year = {2015},
  doi = {10.18653/v1/w15-3215},
  url = {https://doi.org/10.18653/v1/w15-3215}
}

References

19 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mohamed I. Alkanhal, Mohammed A. Al-Badrashiny, Mansour M. Alghamdi, and Abdulaziz O. AlQabbany. 2012. Automatic Stochastic Arabic Spelling Correction with Emphasis on Space Insertions and Deletions. IEEE Transactions on Audio, Speech & Language Processing , 20:2111–2122. ↗
  2. Michele Banko and Eric Brill, 2001. Scaling to very very large corpora for natural language disambiguation. In Proceedings of 39th Annual Meeting of the Association for Computational Linguistics . Toulouse, France.
  3. Chiraz Ben Othmane Zribi and Mohammed Ben Ahmed. 2003. Efficient Automatic Correction of Misspelled Arabic Words Based on Contextual Information. In Proceedings of the Knowledge-Based Intelligent Information and Engineering Systems Conference , Oxford, UK. find
  4. Tim Buckwalter. 2010. Buckwalter Arabic Morphological Analyzer Version 2.0. Linguistic Data Consortium, University of Pennsylvania, 2002. LDC Catalog No.: LDC2004L02. ISBN 1-58563-324-0. find
  5. Andrew Carlson and Ian Fette. 2007. Memory-based context-sensitive spelling correction at web scale. In Proceedings of the IEEE International Conference on Machine Learning and Applications (ICMLA). find
  6. Daniel Dahlmeier and Hwee Tou Ng. 2012. Better evaluation for grammatical error correction. In Proceeding of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. find
  7. Andrew R. Golding and Dan Roth. 1999. A Winnow based approach to context-sensitive spelling correction. Machine Learning , 34(1-3):107–130. ↗
  8. Bassam Haddad and Mustafa Yaseen. 2007. Detection and Correction of Non-Words in Arabic: A Hybrid Approach. International Journal of Computer Processing Of Languages (IJCPOL). ↗
  9. Ahmed Hassan, Sara Noeman, and Hany Hassan. 2008. Language Independent Text Correction using Finite State Automata. In Proceedings of the International Joint Conference on Natural Language Processing (IJCNLP 2008). find
  10. Karen Kukich. 1992. Techniques for Automatically Correcting Words in Text. ACM Computing Surveys , 24(4). ↗
  11. Arfath Pasha and Mohamed Al-Badrashiny and Ahmed El Kholy and Ramy Eskander and Mona Diab and Nizar Habash and Manoj Pooleery and Owen Rambow and Ryan Roth. 2014. MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC)
  12. Alla Rozovskaya, Nizar Habash, Ramy Eskander, Noura Farra, and Wael Salloum. 2014. The Columbia System in the QALB-2014 Shared Task on Arabic Error Correction. In Proceedings of EMNLP Workshop on Arabic Natural Language Processing: QALB Shared Task. ↗
  13. Alla Rozovskaya, Nizar Habash, Wajdi Zaghouani, Ossama Obeid, and Behrang Mohit. 2015. The Second QALB Shared Task on Automatic Text Correction for Arabic. In Proceedings of ACL workshop on Arabic Natural Language Processing . Beijing, China.
  14. Khaled Shaalan, Rana Aref, and Aly Fahmy. 2010. An approach for analyzing and correcting spelling errors for non-native Arabic learners. In Proceedings of Informatics and Systems (IN-FOS). find
  15. Ibrahim, Michael Nawar. 2015. Statistical Arabic Grammar Analyzer. Computational Linguistics and Intelligent Text Processing. Springer International Publishing. 187-200. find
  16. Behrang Mohit, Alla Rozovskaya, Nizar Habash, Wajdi Zaghouani, and Ossama Obeid, 2014. The First shared Task on Automatic Text Correction for Arabic. In Proceedings of EMNLP workshop on Arabic Natural Language Processing . Doha, Qatar. find
  17. Nawar, Michael N., and Moheb M. Ragheb. 2014. Fast and robust arabic error correction system. ANLP 2014. 143. ↗
  18. Wajdi Zaghouani, Behrang Mohit, Nizar Habash, Ossama Obeid, Nadi Tomeh, Alla Rozovskaya, Noura Farra, Sarah Alkuhlani, and Kemal Oflazer. 2014. Large Scale Arabic Error Annotation: Guidelines and Framework. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14) , Reykjavik, Iceland. ↗
  19. Wajdi Zaghouani, Nizar Habash, Houda Bouamor, Alla Rozovskaya, Behrang Mohit, Abeer Heider, and Kemal Oflazer. 2015. Correction Annotation for Non-Native Arabic Texts: Guidelines and Corpus. In Proceedings of the Ninth Linguistic Annotation Workshop , Denver, Colorado, USA. Association for Computational Linguistics. 129-139.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nawar-2015-cufe-qalb-2015-shared-task/