Elamur Research⌕
☰
home / papers / 2014 / CMUQ@QALB-2014: An SMT-based System for Automatic Arabic Error Correction

CMUQ@QALB-2014: An SMT-based System for Automatic Arabic Error Correction

Paper · 2014 · Serena Jeblee, Houda Bouamor, Wajdi Zaghouani et al.

Summary

The paper presents the CMUQ entry to the first QALB-2014 shared task on automatic error correction of Modern Standard Arabic. The task covers text by native speakers, non-native speakers and machines, with errors in spelling, punctuation, splits, merges and word order. The final system is a pipeline:

  • It first swaps each word for the undiacritized form that MADAMIRA picks as most likely ("Feature 14"), which repairs many Hamza and omission errors.
  • Hand-written rules then reattach detached clitics and collapse runs of repeated characters.
  • A Moses phrase-based SMT model then "translates" the still-noisy Arabic into corrected Arabic.

A noisy-channel spelling checker built on the AraComLex word list was tried as a baseline. It hurt performance, so it was dropped. The full pipeline reached F1 62.96% on dev and 65.42% on test with punctuation counted, placing 3rd. The authors describe it as the first MT-based error corrector for Arabic. The work matters because it frames correction as monolingual translation, while cheap deterministic normalization does the easy part first.

Key points

  • Training data: QALB corpus, about 950K tokens. About 306K tokens carry some correction: roughly 169K spelling edits, 99K insertions (mostly punctuation), and only 427 word moves.

  • Dev F1 with punctuation for each configuration:

    ConfigurationDev F1
    Feature 14 alone0.4539
    Spelling-checker baseline0.3810
    + clitic rules0.4983
    + SMT0.5964
    Full pipeline0.6296

    Without punctuation, the full pipeline reaches 0.7473 on dev.

  • SMT setup:

    • Moses with fast_align from cdec and grow-diag symmetrization.
    • A 5-gram KenLM model built on the corrected side of the data.
    • The last 1,000 training sentences held out for tuning.

  • A reordering window of 4 beat the default of 7, as well as 5, 3 and 0, on both BLEU and F1.

  • Error analysis shows the remaining weaknesses:

    • unnecessary-word deletion, merges and splits, and dialect-to-MSA conversion;
    • Hamza errors whose unhamzated form is itself a valid word;
    • some valid edits that score as wrong because the gold standard lacks them.

Interesting ideas

  • I1. Put deterministic, high-precision normalization (a disambiguator's chosen form, clitic rules, character de-duplication) before the statistical model, so that model only learns the residual errors.
  • I2. Mine the training edits to build a lookup from observed wrong forms to their corrections, and fall back to generic edit-distance candidates only for unseen forms.
  • I3. Measure each module's contribution separately and drop a component that lowers the total score, as happened with the spelling checker.
  • I4. Tune distortion or reordering limits to the task: correction rarely moves words, so a short window works better than translation defaults.
  • I5. Report scores both with and without punctuation, and separate wrong edits from plausible edits the gold standard simply lacks.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): Normalize repeated characters and split or glued clitics like "can't"/"cannot" with logged rules before tagging, so the tagger sees cleaner input and every change stays traceable.
  • I2 (learn): Self-induced repair rules can start as a table of attested error→fix pairs taken from silver disagreements, with generic heuristics used only as a backoff.
  • I3 (learn): Run per-stage ablations in evaluation so that a component which looks sensible but hurts end-to-end results is caught and removed.
  • I4 (back): In text→graph→text compression, constrain how far tokens may move relative to the source, keeping regenerated text close to the original.
  • I5 (answer): Score answers on content and surface separately and flag "plausible but unreferenced" outputs, so grading does not punish valid paraphrases.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States539 works · 179,533 citations🎓Carnegie Mellon University QatarDoha · Qatar47 works · 2,773 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksEducation · 203 works

🧭 Topics 2Research subjects and languages the work is about.Machine translation · 1,087 worksSemitic languages · 2,908 works

Cite

Cite as Serena Jeblee, Houda Bouamor, Wajdi Zaghouani and Kemal Oflazer. 2014. CMUQ@QALB-2014: An SMT-based System for Automatic Arabic Error Correction. Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/w14-3618

@inproceedings{jeblee2014cmuq,
  title = {{CMUQ@QALB-2014: An SMT-based System for Automatic Arabic Error Correction}},
  author = {Serena Jeblee, Houda Bouamor, Wajdi Zaghouani and Kemal Oflazer},
  booktitle = {Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP)},
  year = {2014},
  doi = {10.3115/v1/w14-3618},
  url = {https://doi.org/10.3115/v1/w14-3618}
}

References

23 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mohamed I. Alkanhal, Mohamed Al-Badrashiny, Mansour M. Alghamdi, and Abdulaziz O. Al-Qabbany. 2012. Automatic Stochastic Arabic Spelling Correction With Emphasis on Space Insertions and Deletions. IEEE Transactions on Audio, Speech & Language Processing, 20(7):2111–2122. ↗
  2. Mohammed Attia, Pavel Pecina, Younes Samih, Khaled Shaalan, and Josef van Genabith. 2012. Improved Spelling Error Detection and Correction for Arabic. In Proceedings of COLING 2012: Posters, pages 103–112, Mumbai, India.
  3. Daniel Dahlmeier and Hwee Tou Ng. 2012a. A Beam-Search Decoder for Grammatical Error Correction. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 568–578, Jeju Island, Korea. find
  4. Daniel Dahlmeier and Hwee Tou Ng. 2012b. Better Evaluation for Grammatical Error Correction. In NAACL HLT ’12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 568–572. find
  5. Robert Dale. 1997. Computer Assistance in Text Creation and Editing. In Survey of the state of the art in Human Language Technology, chapter 7, pages 235–237. Cambridge University Press. find
  6. Chris Dyer, Jonathan Weese, Hendra Setiawan, Adam Lopez, Ferhan Ture, Vladimir Eidelman, Juri Ganitkevitch, Phil Blunsom, and Philip Resnik. 2010. cdec: A Decoder, Alignment, and Learning Framework for Finite-state and Context-free Translation Models. In Proceedings of the ACL 2010 System Demonstrations, pages 7–12, Uppsala, Sweden.
  7. A. R. Golding and D. Roth. 1999. A Winnow Based Approach to Context-Sensitive Spelling Correction. Machine Learning, 34(1-3):107–130. ↗
  8. Nizar Habash. 2008. Four Techniques for Online Handling of Out-of-Vocabulary Words in Arabic-English Statistical Machine Translation. In Proceedings of ACL-08: HLT, Short Papers, pages 57–60, Columbus, Ohio. find
  9. Bassam Haddad and Mustafa Yaseen. 2007. Detection and Correction of Non-words in Arabic: a Hybrid Approach. International Journal of Computer Processing of Oriental Languages, 20(04):237–257. ↗
  10. Ahmed Hassan, Sara Noeman, and Hany Hassan. 2008. Language Independent Text Correction using Finite State Automata. In Proceedings of the Third International Joint Conference on Natural Language Processing (IJCNLP 2008), pages 913–918, Hyderabad, India. find
  11. Kenneth Heafield, Ivan Pouzyrevsky, Jonathan H. Clark, and Philipp Koehn. 2013. Scalable Modfied Kneser-Ney Language Model Estimation. In In Proceedings of the Association for Computational Linguistics, Sofia, Bulgaria. find
  12. Philipp Koehn, Hieu Hoang, Alexandra Birch, Christopher Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Christopher Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open Source Toolkit for Statistical Machine Translation. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics Companion Volume Proceedings of the Demo and Poster Sessions, pages 177–180, Prague, Czech Republic. find
  13. Karen Kukich. 1992. Techniques for Automatically Correcting Words in Text. ACM Computing Surveys (CSUR), 24(4):377–439. ↗
  14. Behrang Mohit, Alla Rozovskaya, Nizar Habash, Wajdi Zaghouani, and Ossama Obeid. 2014. The First QALB Shared Task on Automatic Text Correction for Arabic. In Proceedings of EMNLP Workshop on Arabic Natural Language Processing, Doha, Qatar, October.
  15. Franz Josef Och and Hermann Ney. 2003. A Systematic Comparison of Various Statistical Alignment Models. In Computational Linguistics, page 1951. ↗
  16. Kemal Oflazer. 1996. Error-Tolerant Finite-State Recognition with Applications to Morphological Analysis and Spelling Correction. Computational Linguistics, 22(1):73–89. ↗
  17. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A Method for Automatic Evaluation of Machine Translation. In Proceedings of the Association for Computational Linguistics, Philadelphia, Pennsylvania.
  18. Arfath Pasha, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 1094–1101, Reykjavik, Iceland.
  19. Khaled Shaalan, Amin Allam, and Abdallah Gomah. 2003. Towards Automatic Spell Checking for Arabic. In Proceedings of the 4th Conference on Language Engineering, Egyptian Society of Language Engineering (ELSE), Cairo, Egypt. find
  20. Khaled Shaalan, Rana Aref, and Aly Fahmy. 2010. An Approach for Analyzing and Correcting Spelling Errors for Non-native Arabic Learners. In Proceedings of The 7th International Conference on Informatics and Systems, INFOS2010, the special track on Natural Language Processing and Knowledge Mining, pages 28–30, Cairo, Egypt. find
  21. Khaled Shaalan, Mohammed Attia, Pavel Pecina, Younes Samih, and Josef van Genabith. 2012. Arabic Word Generation and Modelling for Spell Checking. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC-2012), pages 719–725, Istanbul, Turkey.
  22. Wajdi Zaghouani, Behrang Mohit, Nizar Habash, Ossama Obeid, Nadi Tomeh, Alla Rozovskaya, Noura Farra, Sarah Alkuhlani, and Kemal Oflazer. 2014. Large Scale Arabic Error Annotation: Guidelines and Framework. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), Reykjavik, Iceland. ↗
  23. Chiraz Zribi and Mohammed Ben Ahmed. 2003. Efficient Automatic Correction of Misspelled Arabic Words Based on Contextual Information. In Proceedings of the Knowledge-Based Intelligent Information and Engineering Systems Conference, pages 770–777, Oxford, UK. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/jeblee-2014-cmuq-qalb-2014-smt-based/