Elamur Research⌕
☰
home / papers / 2015 / Correction Annotation for Non-Native Arabic Texts: Guidelines and Corpus

Correction Annotation for Non-Native Arabic Texts: Guidelines and Corpus

Paper · 2015 · Wajdi Zaghouani, Nizar Habash, Houda Bouamor et al.

Summary

Learners of Arabic as a second language make errors that differ from those of native writers. Error-corrected L2 Arabic data, however, has been small, unannotated or not publicly available. The authors extend the QALB project, which built a large manually corrected corpus of native (L1) Arabic, to learner essays. They take two freely available learner collections, ALWC and ALC, totalling about 189K words. They then adapt their L1 correction guidelines with rules aimed at typical L2 problems, refining them iteratively over annotation rounds. The guidelines ask for minimal edits that make text accurate and fluent while leaving style untouched. They also fix a preference order that favours the least intrusive fix. Agreement is measured as WER between annotators' corrected outputs. L2 text is harder than L1: it needs more changes, and annotators diverge more, often because a sentence allows several readings. Even so, consistency stays high after a second round. The result is described as the largest corrected L2 Arabic corpus to date. A portion of it supplies data for QALB's second shared task, which concerns correcting Arabic errors automatically.

Key points

  • The corpus combines 51K words from ALWC (converted from PDF and checked by hand) and 139K words from ALC (92 learners in Saudi Arabia, split by level), about 189K words in total.
  • The guidelines define seven error categories, including spelling, word choice, morphology, syntax and proper names. Fixes follow a priority order: inflection, then cliticization, derivation, preposition, word choice and finally word insertion.
  • Annotators changed 37.64% of words in L2 raw text versus 24.45% in L1. First-round agreement (as WER) was 14.67% for L2 versus 3.80% for L1.
  • The second round starts from the first-round output, and there L2 disagreement drops to 3.35%. By subcorpus, ALC scores 13.56% then 3.13%, and ALWC scores 19.12% then 4.20%. ALWC needed the most edits (51.39%).
  • A comparison of 5K-word samples shows clear differences between the two groups:
    • Word order, missing word and redundant word errors are mainly L2.
    • Punctuation, Hamza spelling and agreement errors appear in both.

Interesting ideas

  • I1. Rank candidate corrections by how intrusive they are and prefer the smallest kind that works, which reduces arbitrary rewrites and makes annotators converge.
  • I2. Measure agreement twice: first on independent corrections of raw text, then on corrections that start from the first-round output. This separates hard interpretive disagreement from residual noise.
  • I3. Restrict edits to real errors and explicitly forbid style improvement or normalising informal register, which curbs over-correction.
  • I4. Disagreement often comes from genuinely different readings of a confusing phrase, not from careless annotators, so one gold correction can misrepresent the data.
  • I5. Compare top error-type rankings across two populations on equally sized samples to see which errors are shared and which are population-specific.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): self-induced repair rules could be tried in a fixed cost order (morphology fix before word insertion), so every repair is explainable as the cheapest adequate one.
  • I2 (learn): silver-data agreement among witness parsers could be reported in two stages, showing how much divergence remains once an initial consensus tree is shared.
  • I3 (tokens): a pre-parse normaliser for noisy English input should fix only what breaks tokenisation or tagging, leaving colloquial phrasing intact so later stages see the author's actual wording.
  • I4 (world): when a garbled sentence supports several interpretations, the text world could keep alternative entity/event readings flagged as ambiguous instead of committing to one.
  • I5 (learn): evaluation could rank parser or frame errors by type per genre (fables vs school tests) to target fixes where a genre's profile differs.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States539 works · 179,533 citations🎓Columbia UniversityNew York · United States241 works · 54,053 citations🎓New York University Abu DhabiAbu Dhabi · United Arab Emirates122 works · 4,602 citations🎓Qatar UniversityDoha · Qatar48 works · 1,026 citations🎓Carnegie Mellon University QatarDoha · Qatar47 works · 2,773 citations

 Fields, topics and concepts

📚 Shelves 8The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksPsychology · 1,552 worksDevelopmental and Educational Psychology · 499 worksEducation · 203 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Wajdi Zaghouani, Nizar Habash, Houda Bouamor, Alla Rozovskaya, Behrang Mohit, Abeer Heider and Kemal Oflazer. 2015. Correction Annotation for Non-Native Arabic Texts: Guidelines and Corpus. Proceedings of the 9th Linguistic Annotation Workshop.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/w15-1614

@inproceedings{zaghouani2015correction,
  title = {{Correction Annotation for Non-Native Arabic Texts: Guidelines and Corpus}},
  author = {Wajdi Zaghouani, Nizar Habash, Houda Bouamor, Alla Rozovskaya, Behrang Mohit, Abeer Heider and Kemal Oflazer},
  booktitle = {Proceedings of the 9th Linguistic Annotation Workshop},
  year = {2015},
  doi = {10.3115/v1/w15-1614},
  url = {https://doi.org/10.3115/v1/w15-1614}
}

References

34 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ghazi Abuhakema, Reem Faraj, Anna Feldman, and Eileen Fitzpatrick. 2008. Annotating an Arabic Learner Corpus for Error. In Proceedings of The sixth international conference on Language Resources and Evaluation, LREC 2008, Marrakech, Morocco. find
  2. Abdullah Alfaifi and Eric Atwell. 2012. Arabic Learner Corpora (ALC): A Taxonomy of Coding Errors. In The 8th International Computing Conference in Arabic (ICCA 2012), Cairo, Egypt. ↗
  3. Daniel Dahlmeier, Hwee Tou Ng, and Mei Wu Wu. 2013. Building a Large Annotated Corpus of Learner English: The NUS Corpus of Learner English. In Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications, pages 22–31, Atlanta, Georgia.
  4. Markus Dickinson and Scott Ledbetter. 2012. Annotating Errors in a Hungarian Learner Corpus. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LRECâĂŹ12), Istanbul, Turkey. find
  5. Ramy Eskander, Nizar Habash, Owen Rambow, and Nadi Tomeh. 2013. Processing Spontaneous Orthography. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), Atlanta, GA. find
  6. Samira Farwaneh and Mohammed Tamimi. 2012. Arabic Learners Written Corpus: A Resource for Research and Learning. The Center for Educational Resources in Culture, Language and Literacy. find
  7. Michael Gamon, Jianfeng Gao, Chris Brockett, Alexandre Klementiev, William B. Dolan, Dmitriy Belenko, and Lucy Vanderwende. 2008. Using Contextual Speller Techniques and Language Modeling for ESL Error Correction. In Third International Joint Conference on Natural Language Processing, IJCNLP, pages 449–456, Hyderabad, India.
  8. Sylviane Granger. 2003. Error-Tagged Learner Corpora and CALL: A Promising Synergy. CALICO, 20(3):465–480. find
  9. Nizar Habash, Abdelhadi Soudi, and Tim Buckwalter. 2007. On Arabic Transliteration. In A. van den Bosch and A. Soudi, editors, Arabic Computational Morphology: Knowledge-based and Empirical Methods. Springer. ↗
  10. Nizar Habash, Owen Rambow, Mona Diab, and Reem Kanjawi-Faraj. 2008. Guidelines for Annotation of Arabic Dialectness. In Proceedings of the LREC Workshop on HLT & NLP within the Arabic world, Marrakech, Morocco. find
  11. Nizar Habash, Mona Diab, and Owen Rabmow. 2012. Conventional Orthography for Dialectal Arabic. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LRECâĂŹ12), Istanbul, Turkey.
  12. Nizar Habash, Ryan Roth, Owen Rambow, Ramy Eskander, and Nadi Tomeh. 2013. Morphological Analysis and Disambiguation for Dialectal Arabic. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), Atlanta, GA.
  13. Nizar Habash. 2010. Introduction to Arabic Natural Language Processing. Morgan & Claypool Publishers. ↗
  14. Björn Hammarberg and Gintaré Grigonyté. 2014. Non-Native Writers’ Errors a Challenge to a Spell-Checker. In 1st Nordic workshop on evaluation of spellchecking and proofing tools (NorWEST2014), Uppsala, Sweden. find
  15. Jirka Hana, Alexandr Rosen, Svatava Škodová, and Barbora Štindlová. 2010. Error-tagged Learner Corpus of Czech. In Proceedings of The Fourth Linguistic Annotation Workshop (LAW IV), Uppsala. find
  16. Haslina Hassan and Nuraihan Mat Daud. 2011. Corpus Analysis of Conjunctions: Arabic Learners Difficulties with Collocations. In Proceedings of the Workshop on Arabic Corpus Linguistics (WACL), Lancaster, UK. find
  17. Emi Izumi, Kiyotaka Uchimoto, and Hitoshi Isahara. 2004. The NICT JLE Corpus Exploiting the Language Learners’ Speech Database for Research and Education. International Journal of The Computer, the Internet and Management, 12(2):119–125, May. find
  18. Mustafa Jarrar, Nizar Habash, Diyam Akra, and Nasser Zalmout. 2014. Building a Corpus for Palestinian Arabic: a Preliminary Study. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 18–27, Doha, Qatar.
  19. Serena Jeblee, Houda Bouamor, Wajdi Zaghouani, and Kemal Oflazer. 2014. Cmuq@ qalb-2014: An smt-based system for automatic arabic error correction. ANLP 2014, page 137.
  20. Claudia Leacock, martin Chodorow, Michael Gamon, and Joel Tetreault. 2010. Automated Grammatical Error Detection for Language Learners. Synthesis Lectures on Human Language Technologies, 3(1):1–134. ↗
  21. Behrang Mohit, Alla Rozovskaya, Nizar Habash, Wajdi Zaghouani, and Ossama Obeid. 2014. The First QALB Shared Task on Automatic Text Correction for Arabic. In Proceedings of EMNLP Workshop on Arabic Natural Language Processing, Doha, Qatar, October.
  22. Ryo Nagata, Atsuo Kawai, Koichiro Morihiro, and Naoki Isu. 2006. A Feedback-Augmented Method for Detecting Errors in the Writing of Learners of English. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 241–248, Sydney, Australia.
  23. Ossama Obeid, Wajdi Zaghouani, Behrang Mohit, Nizar Habash, Kemal Oflazer, and Nadi Tomeh. 2013. A Web-based Annotation Framework For Large-Scale Text Correction. In The Companion Volume of the Proceedings of IJCNLP 2013: System Demonstrations, Nagoya, Japan, October. find
  24. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, PA.
  25. Arfath Pasha, Mohamed Al-Badrashiny, Ahmed El Kholy, Ramy Eskander, Mona Diab, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic. In In Proceedings of the 9th International Conference on Language Resources and Evaluation, Reykjavik, Iceland.
  26. Alla Rozovskaya and Dan Roth. 2010. Annotating ESL Errors: Challenges and Rewards. In NAACL Workshop on Innovative Use of NLP for Building Educational Applications, Los Angeles, CA. find
  27. Alla Rozovskaya, Nizar Habash, Ramy Eskander, Noura Farra, and Wael Salloum. 2014. The columbia system in the qalb-2014 shared task on arabic error correction. In Workshop on Arabic Natural Language Processing, EMNLP, page 160. ↗
  28. Wael Salloum and Nizar Habash. 2013. Dialectal Arabic to English Machine Translation: Pivoting through Modern Standard Arabic. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), Atlanta, GA.
  29. Thomas Schmidt. 2010. Linguistic Tool Development between Community Practices and Technology Standards. In Proceedings of the LREC Workshop Language Resource and Language Technology Standards State of the Art, Emerging Needs, and Future Developments. find
  30. Joel Tetreault and Martin Chodorow. 2008. Native Judgments of Non-Native Usage: Experiments in Preposition Error Detection. In Coling 2008: Proceedings of the workshop on Human Judgements in Computational Linguistics, pages 24–32, Manchester, UK. find
  31. Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 2011. A New Dataset and Method for Automatically Grading ESOL Texts. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 180–189, Portland, Oregon, USA.
  32. Wajdi Zaghouani, Nizar Habash, and Behrang Mohit. 2014a. The Qatar Arabic Language Bank Guidelines. Technical Report CMU-CS-QTR-124, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, September. find
  33. Wajdi Zaghouani, Behrang Mohit, Nizar Habash, Ossama Obeid, Nadi Tomeh, Alla Rozovskaya, Noura Farra, Sarah Alkuhlani, and Kemal Oflazer. 2014b. Large Scale Arabic Error Annotation: Guidelines and Framework. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), Reykjavik, Iceland, May. ↗
  34. Inès Zribi, Rahma Boujelbane, Abir Masmoudi, Mariem Ellouze, Lamia Belguith, and Nizar Habash. 2014. A Conventional Orthography for Tunisian Arabic. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 2355–2361, Reykjavik, Iceland.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zaghouani-2015-correction-annotation-non-native-arabic/