Elamur Research⌕
☰
home / papers / 2015 / UMMU@QALB-2015 Shared Task: Character and Word level SMT pipeline for Automatic Error Correction of Arabic Text

UMMU@QALB-2015 Shared Task: Character and Word level SMT pipeline for Automatic Error Correction of Arabic Text

Paper · 2015 · Fethi Bougares and Houda Bouamor

Summary

The paper addresses automatic correction of spelling and editing errors in Arabic text for the QALB-2015 shared task. The task had two tracks: comments on Aljazeera articles written by native speakers (L1), and sentences written by learners of Arabic (L2). The authors frame correction as monolingual translation from erroneous to corrected text and build phrase-based SMT systems with MOSES. They compare a word-level and a character-level system, each trained either on raw input or on input already corrected by MADAMIRA. The language model uses the corrected QALB side plus Arabic Gigaword sentences chosen by cross-entropy data selection. The best pipeline has three stages: MADAMIRA's corrections, then a character-level SMT pass, then a word-level SMT pass over the re-joined output. It reaches F1 69.42 on the 2014 test set, above the previous best of 67.91. On the official 2015 test it ranked first in L2 and second in L1. The work shows that error correction at several granularities, applied in sequence, is complementary.

Key points

  • Data: Alj-train-2014 has 19,411 sentences (1.1M tokens), while L2-train-2015 has only 310 sentences (46.3k tokens). Systems were tuned on Alj-dev-2014 and L2-dev-2015.
  • The MADAMIRA-only baseline is very precise but recalls little: P 77.47, R 32.10, F1 45.40 on Alj-dev-2014.
  • The character-level SMT beats the word-level one on raw data (F1 66.12 vs 61.31 on dev). Adding MADAMIRA pre-processing lifts the character system to 67.45 on dev and 40.12 on L2-dev.
  • Order matters when chaining the systems:
    • Word-then-character ("top-down") scores 66.48 F1 on Alj-test-2014.
    • Character-then-word ("bottom-up") scores 69.42 on Alj-test-2014 and 43.13 on L2-dev.
  • Official primary run UMMU-1:
    • F1 71.10 on L1-test-2015 (P 70.28, R 71.93), ranked 2nd of 12.
    • F1 41.20 on L2-test-2015, ranked 1st of 10.
  • Language models: a 4-gram LM for words and a 9-gram LM for characters, built with SRILM and Kneser-Ney smoothing. Gigaword data was selected with XenC by minimising perplexity on the dev set.

Interesting ideas

  • I1. Cast noisy-to-clean normalisation as translation between two versions of the same language, with a noisy-channel decoder learning edits from aligned pairs instead of hand-written rules.
  • I2. Run fine-grained repairs first, then a coarser pass over their output: character fixes clean up word forms so the word-level model sees recognisable tokens, and reversing the order was clearly worse.
  • I3. Start from a high-precision, low-recall tool (MADAMIRA) and let a learned system add recall on top. Each stage improved F1, showing the stages are complementary.
  • I4. Diagnose combinations through the precision/recall trade-off: the word stage raised recall but cut precision, which points to where a smarter combination should intervene.
  • I5. Choose extra language-model data by cross-entropy against in-domain dev text rather than using a whole large corpus.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): A small character-level "translator" trained on misspelled/clean pairs could normalise typos before tagging and lemmatisation, so downstream lemmas stay stable on messy input.
  • I2 (tokens): Order Mova's repair passes from orthographic to token to phrase level, and record each pass's edits so the final correction stays explainable.
  • I3 (learn): Self-induced repair rules could be layered the same way, with a conservative rule set applied first and learned corrections after, each layer judged by whether it adds F1 without eroding precision.
  • I4 (learn): When agreeing silver data across witness parsers, report precision and recall shifts per added witness, not just a single score, to see which witness adds noise.
  • I5 (learn): The same selection could filter large unlabeled English corpora toward the style of school tests or fables before generating silver training trees.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States539 works · 179,533 citations🎓Le Mans UniversitéLe Mans · France15 works · 31,097 citations🎓Carnegie Mellon University QatarDoha · Qatar47 works · 2,773 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Machine translation · 1,087 worksSemitic languages · 2,908 works

Cite

Cite as Fethi Bougares and Houda Bouamor. 2015. UMMU@QALB-2015 Shared Task: Character and Word level SMT pipeline for Automatic Error Correction of Arabic Text. Proceedings of the Second Workshop on Arabic Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w15-3221

@inproceedings{bougares2015ummu,
  title = {{UMMU@QALB-2015 Shared Task: Character and Word level SMT pipeline for Automatic Error Correction of Arabic Text}},
  author = {Fethi Bougares and Houda Bouamor},
  booktitle = {Proceedings of the Second Workshop on Arabic Natural Language Processing},
  year = {2015},
  doi = {10.18653/v1/w15-3221},
  url = {https://doi.org/10.18653/v1/w15-3221}
}

References

26 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mohamed I. Alkanhal, Mohamed Al-Badrashiny, Mansour M. Alghamdi, and Abdulaziz O. Al-Qabbany. 2012. Automatic Stochastic Arabic Spelling Correction With Emphasis on Space Insertions and Deletions. IEEE Transactions on Audio, Speech & Language Processing, 20(7):2111–2122. ↗
  2. Mohammed Attia, Pavel Pecina, Younes Samih, Khaled Shaalan, and Josef van Genabith. 2012. Improved Spelling Error Detection and Correction for Arabic. In Proceedings of COLING 2012: Posters, pages 103–112, Mumbai, India.
  3. Abdelmajid Ben Hamadou. 1994. The Phases of Computational Analysis of Arabic Towards Detecting and Correcting Errors. In Proceedings of the Second Conference for Arabization of Computers, in Arabic. find
  4. Daniel Dahlmeier and Hwee Tou Ng. 2012. A Beam-Search Decoder for Grammatical Error Correction. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 568–578, Jeju Island, Korea. find
  5. A. R. Golding and D. Roth. 1999. A Winnow Based Approach to Context-Sensitive Spelling Correction. Machine Learning, 34(1-3):107–130. ↗
  6. Nizar Habash, Owen Rambow, Mona Diab, and Reem Kanjawi-Faraj. 2008. Guidelines for Annotation of Arabic Dialectness. In Proceedings of the LREC Workshop on HLT & NLP within the Arabic world, Marrakech, Morocco. find
  7. Nizar Habash. 2008. Four Techniques for Online Handling of Out-of-Vocabulary Words in Arabic-English Statistical Machine Translation. In Proceedings of ACL-08: HLT, Short Papers, pages 57– 60, Columbus, Ohio. find
  8. Nizar Habash. 2010. Introduction to Arabic Natural Language Processing, volume 3. Morgan & Claypool Publishers. find
  9. Bassam Haddad and Mustafa Yaseen. 2007. Detection and Correction of Non-words in Arabic: a Hybrid Approach. International Journal of Computer Processing of Oriental Languages, 20(04):237–257. ↗
  10. Ahmed Hassan, Sara Noeman, and Hany Hassan. 2008. Language Independent Text Correction using Finite State Automata. In Proceedings of the Third International Joint Conference on Natural Language Processing (IJCNLP 2008), pages 913– 918, Hyderabad, India. find
  11. Serena Jeblee, Houda Bouamor, Wajdi Zaghouani, and Kemal Oflazer. 2014. Cmuq@qalb-2014: An smt-based system for automatic arabic error correction. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 137–142, Doha, Qatar, October. Association for Computational Linguistics.
  12. P. Koehn, H. Hoang, A. Birch, C. Callison-Burch, M. Federico, N. Bertoldi, B. Cowan, W. Shen, C. Moran, R. Zens, et al. 2007. Moses: Open source toolkit for statistical machine translation. In Annual meeting-association for computational linguistics, volume 45, page 2. find
  13. Karen Kukich. 1992. Techniques for Automatically Correcting Words in Text. ACM Computing Surveys (CSUR), 24(4):377–439. ↗
  14. Behrang Mohit, Alla Rozovskaya, Nizar Habash, Wajdi Zaghouani, and Ossama Obeid. 2014. The First QALB Shared Task on Automatic Text Correction for Arabic. In Proceedings of EMNLP Workshop on Arabic Natural Language Processing, Doha, Qatar, October.
  15. Kemal Oflazer. 1996. Error-Tolerant Finite-State Recognition with Applications to Morphological Analysis and Spelling Correction. Computational Linguistics, 22(1):73–89. ↗
  16. Arfath Pasha, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 1094–1101, Reykjavik, Iceland.
  17. Anthony Rousseau. 2013. Xenc: an open-source tool for data selection in natural language processing. Prague Bulletin of Mathematical Linguistics, 100:73–82. ↗
  18. Alla Rozovskaya, Houda Bouamor, Nizar Habash, Wajdi Zaghouani, Ossama Obeid, and Behrang Mohit. 2015. The Second QALB Shared Task on Automatic Text Correction for Arabic. In Proceedings of ACL Workshop on Arabic Natural Language Processing, Beijing, China, July.
  19. Khaled Shaalan, Amin Allam, and Abdallah Gomah. 2003. Towards Automatic Spell Checking for Arabic. In Proceedings of the 4th Conference on Language Engineering, Egyptian Society of Language Engineering (ELSE), Cairo, Egypt. find
  20. Khaled Shaalan, Rana Aref, and Aly Fahmy. 2010. An Approach for Analyzing and Correcting Spelling Errors for Non-native Arabic Learners. In Proceedings of The 7th International Conference on Informatics and Systems, INFOS2010, the special track on Natural Language Processing and Knowledge Mining, pages 28–30, Cairo, Egypt. find
  21. Khaled Shaalan, Mohammed Attia, Pavel Pecina, Younes Samih, and Josef van Genabith. 2012. Arabic Word Generation and Modelling for Spell Checking. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC-2012), pages 719–725, Istanbul, Turkey.
  22. Khaled F Shaalan. 2005. Arabic GramCheck: A Grammar Checker for Arabic. Software: Practice and Experience, 35(7):643–665. ↗
  23. Andreas Stolcke, Jing Zheng, Wen Wang, and Victor Abrash. 2011. SRILM at sixteen: Update and outlook. In Proceedings of the Automatic Speech Recognition and Understanding Workshop (ASRU), Décembre. find
  24. Wajdi Zaghouani, Behrang Mohit, Nizar Habash, Ossama Obeid, Nadi Tomeh, Alla Rozovskaya, Noura Farra, Sarah Alkuhlani, and Kemal Oflazer. 2014. Large Scale Arabic Error Annotation: Guidelines and Framework. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), Reykjavik, Iceland. ↗
  25. Wajdi Zaghouani, Nizar Habash, Houda Bouamor, Alla Rozovskaya, Behrang Mohit, Abeer Heider, and Kemal Oflazer. 2015. Correction annotation for non-native arabic texts: Guidelines and corpus. In Proceedings of The 9th Linguistic Annotation Workshop, pages 129–139, Denver, Colorado, USA, June. Association for Computational Linguistics.
  26. Chiraz Zribi and Mohammed Ben Ahmed. 2003. Efficient Automatic Correction of Misspelled Arabic Words Based on Contextual Information. In Proceedings of the Knowledge-Based Intelligent Information and Engineering Systems Conference, pages 770–777, Oxford, UK. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/bougares-2015-ummu-qalb-2015-shared-task/