Elamur Research⌕
☰
home / papers / 2015 / QCMUQ@QALB-2015 Shared Task: Combining Character level MT and Error-tolerant Finite-State Recognition for Arabic Spelling Correction

QCMUQ@QALB-2015 Shared Task: Combining Character level MT and Error-tolerant Finite-State Recognition for Arabic Spelling Correction

Paper · 2015 · Houda Bouamor, Hassan Sajjad, Nadir Durrani et al.

Summary

This paper describes the joint CMU-Q and QCRI entry to the QALB-2015 shared task, which asks systems to fix errors in Modern Standard Arabic text. The test covers two genres: Aljazeera news comments (L1) and sentences written by non-native students of Arabic (L2). The errors vary widely, from split and merged words to letter-level slips and punctuation, so the authors built a cascade of specialised correctors. Each one takes the previous one's output.

The stages are:

  • MADAMIRA's Alif/Ya fixes
  • hand-written rules
  • phrase-based and character-level SMT models built with Moses
  • an error-tolerant finite-state recognizer that proposes dictionary words within a small edit distance, ranked by a language model

The best chain is Morph+CBMT+Rules+EFST. On the official 2015 test it scored F1 68.12 on L1 and 38.90 on L2, ranking 5th and 2nd respectively. The work shows that cheap, complementary correctors, each aimed at one error type, can add up to a strong system for a language with few resources.

Key points

  • Training and evaluation use QALB corpus data and the m2Scorer. Language models come from GigaWord Arabic plus News Commentary (about 32M sentences, ~1.7B tokens), but only a random 60% was used because of compute limits.
  • The dictionary is a 9-million-word Arabic list generated from the AraComLex transducer (Attia et al., 2012). It was compiled into an FST with Foma for error-tolerant lookup.
  • The CBMT model treats each sentence as a sequence of characters. It uses MGIZA alignment, grow-diagonal-final symmetrization and 5-gram KenLM models.
    • It was the strongest single module: F1 64.07 on the 2014 test and 39.07 on L2 dev.
  • MADAMIRA corrections alone are very precise (P 78.33) but have low recall (31.27).
    • Adding Morph in front of PBMT raised F1 by 3.22 points.
  • The full chain reached F1 68.42 on the 2014 test (P 70.14, R 66.79) and 44.02 on L2 dev, against 20.28 for the Morph baseline.
    • Official 2015 test scores: L1 P 71.39, R 65.13; L2 P 50.37, R 31.68.

Interesting ideas

  • I1. Rather than one corrector for everything, the authors chain narrow experts, each aimed at one error class, and keep a combination only if the dev score rises. This makes every fix attributable to a specific module.
  • I2. Spelling repair is framed as translation over characters, with spaces as ordinary symbols, so a single model learns both word splits and merges.
  • I3. The error-tolerant finite-state recognizer finds dictionary words within a small edit distance (insertions, deletions, substitutions, transpositions), and a language model then picks among the candidates.
  • I4. A data analysis revealed that one conjunction accounts for 95% of merge errors, which justified a targeted rule instead of a general model.
  • I5. High-precision, low-recall modules are placed early, and higher-recall ones run on their output, which improves both precision and recall together.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): Mova's input normalisation could be a sequence of small, separately scored repair passes for spacing, casing, elongation and typos, each logging what it changed.
  • I2 (tokens): a character-level noisy-to-clean model could repair glued or broken English tokens before tagging, rather than relying only on tokenizer heuristics.
  • I3 (tokens): Mova's lexicon could be compiled into an FST so that out-of-vocabulary words get explainable nearest-word candidates before lemmatisation.
  • I4 (learn): self-induced repair rules could be mined in the same way, by counting which few patterns dominate parser failures and promoting those to explicit rules.
  • I5 (learn): when merging witness parsers for silver data, conservative high-precision rules could be applied first, leaving the remaining arcs to broader models.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States539 works · 179,533 citations🎓Carnegie Mellon University QatarDoha · Qatar47 works · 2,773 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksEducation · 203 works

🧭 Topics 3Research subjects and languages the work is about.Machine translation · 1,087 worksSemitic languages · 2,908 worksGrammatical error correction & learner language · 156 works

Cite

Cite as Houda Bouamor, Hassan Sajjad, Nadir Durrani and Kemal Oflazer. 2015. QCMUQ@QALB-2015 Shared Task: Combining Character level MT and Error-tolerant Finite-State Recognition for Arabic Spelling Correction. Proceedings of the Second Workshop on Arabic Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w15-3217

@inproceedings{bouamor2015qcmuq,
  title = {{QCMUQ@QALB-2015 Shared Task: Combining Character level MT and Error-tolerant Finite-State Recognition for Arabic Spelling Correction}},
  author = {Houda Bouamor, Hassan Sajjad, Nadir Durrani and Kemal Oflazer},
  booktitle = {Proceedings of the Second Workshop on Arabic Natural Language Processing},
  year = {2015},
  doi = {10.18653/v1/w15-3217},
  url = {https://doi.org/10.18653/v1/w15-3217}
}

References

25 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mohammed Attia, Pavel Pecina, Younes Samih, Khaled Shaalan, and Josef van Genabith. 2012. Improved Spelling Error Detection and Correction for Arabic. In Proceedings of COLING 2012: Posters, pages 103–112, Mumbai, India.
  2. Daniel Dahlmeier and Hwee Tou Ng. 2012. Better Evaluation for Grammatical Error Correction. In NAACL HLT ’12 Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 568–572. find
  3. Mona Diab, Mohammed Attia, and Al-Badrashiny Mohamed. 2014. GWU-HASP: Hybrid Arabic Spelling and Punctuation Corrector. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), Doha, Qatar.
  4. Nadir Durrani, Yaser Al-Onaizan, and Abraham Ittycheriah. 2014a. Improving Egyptian-to-English SMT by mapping Egyptian into MSA. In Computational Linguistics and Intelligent Text Processing, pages 271–282, Khatmandu, Nepal. Springer Berlin Heidelberg. find
  5. Nadir Durrani, Barry Haddow, Philipp Koehn, and Kenneth Heafield. 2014b. Edinburgh’s phrase-based machine translation systems for WMT-14. In Proceedings of the ACL 2014 Ninth Workshop on Statistical Machine Translation, WMT ’14, pages 97–104, Baltimore, MD, USA. find
  6. Nadir Durrani, Hassan Sajjad, Hieu Hoang, and Philipp Koehn. 2014c. Integrating an unsupervised transliteration model into statistical machine translation. In Proceedings of the 15th Conference of the European Chapter of the ACL, EACL ’14, Gothenburg, Sweden. find
  7. Qin Gao and Stephan Vogel. 2008. Parallel Implementations of Word Alignment Tool. In Software Engineering, Testing, and Quality Assurance for Natural Language Processing, pages 49–57. Association for Computational Linguistics.
  8. Bassam Haddad and Mustafa Yaseen. 2007. Detection and Correction of Non-words in Arabic: a Hybrid Approach. International Journal of Computer Processing of Oriental Languages, 20(04):237–257. ↗
  9. Ahmed Hassan, Sara Noeman, and Hany Hassan. 2008. Language Independent Text Correction using Finite State Automata. In Proceedings of the Third International Joint Conference on Natural Language Processing (IJCNLP 2008), pages 913– 918, Hyderabad, India. find
  10. Kenneth Heafield. 2011. KenLM: Faster and smaller language model queries. In Proceedings of the Workshop on Statistical Machine Translation (WMT’11), Edinburgh, UK.
  11. Mans Hulden. 2009. Foma: A finite-state compiler and library. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics: Demonstrations Session, EACL ’09, pages 29–32, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  12. Serena Jeblee, Houda Bouamor, Wajdi Zaghouani, and Kemal Oflazer. 2014. Cmuq@qalb-2014: An smt-based system for automatic arabic error correction. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 137–142, Doha, Qatar, October. Association for Computational Linguistics.
  13. Philipp Koehn, Hieu Hoang, Alexandra Birch, Christopher Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Christopher Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open Source Toolkit for Statistical Machine Translation. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics Companion Volume Proceedings of the Demo and Poster Sessions, pages 177–180, Prague, Czech Republic. find
  14. Karen Kukich. 1992. Techniques for Automatically Correcting Words in Text. ACM Computing Surveys (CSUR), 24(4):377–439. ↗
  15. Behrang Mohit, Alla Rozovskaya, Nizar Habash, Wajdi Zaghouani, and Ossama Obeid. 2014. The First QALB Shared Task on Automatic Text Correction for Arabic. In Proceedings of EMNLP Workshop on Arabic Natural Language Processing, Doha, Qatar, October.
  16. Franz Josef Och and Hermann Ney. 2003. A Systematic Comparison of Various Statistical Alignment Models. In Computational Linguistics, page 1951. ↗
  17. Kemal Oflazer. 1996. Error-tolerant Finite-state Recognition with Applications to Morphological Analysis and Spelling Correction. Comput. Linguist., 22(1):73–89, March. ↗
  18. Arfath Pasha, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan M Roth. 2014. MADAMIRA: A fast, comprehensive tool for morphological analysis and disambiguation of Arabic. In Proceedings of the Language Resources and Evaluation Conference, LREC ’14, pages 1094– 1101, Reykjavik, Iceland.
  19. Alla Rozovskaya, Kai-Wei Chang, Mark Sammons, and Dan Roth. 2013. The University of Illinois System in the CoNLL-2013 Shared Task. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning: Shared Task, pages 13–19, Sofia, Bulgaria, August. Association for Computational Linguistics. find
  20. Alla Rozovskaya, Nizar Habash, Ramy Eskander, Noura Farra, and Wael Salloum. 2014. The columbia system in the qalb-2014 shared task on arabic error correction. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 160–164, Doha, Qatar, October. Association for Computational Linguistics. ↗
  21. Alla Rozovskaya, Houda Bouamor, Nizar Habash, Wajdi Zaghouani, Ossama Obeid, and Behrang Mohit. 2015. The Second QALB Shared Task on Automatic Text Correction for Arabic. In Proceedings of ACL Workshop on Arabic Natural Language Processing, Beijing, China, July.
  22. Hassan Sajjad, Kareem Darwish, and Yonatan Belinkov. 2013a. Translating dialectal Arabic to English. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), ACL ’13, pages 1–6, Sofia, Bulgaria. find
  23. Hassan Sajjad, Francisco Guzmn, Preslav Nakov, Ahmed Abdelali, Kenton Murray, Fahad Al Obaidli, and Stephan Vogel. 2013b. QCRI at IWSLT 2013: Experiments in Arabic-English and English-Arabic spoken language translation. In Proceedings of the 10th International Workshop on Spoken Language Technology, IWSLT ’13, Heidelberg, Germany. find
  24. Khaled Shaalan, Amin Allam, and Abdallah Gomah. 2003. Towards Automatic Spell Checking for Arabic. In Proceedings of the 4th Conference on Language Engineering, Egyptian Society of Language Engineering (ELSE), Cairo, Egypt. find
  25. Wajdi Zaghouani, Behrang Mohit, Nizar Habash, Ossama Obeid, Nadi Tomeh, Alla Rozovskaya, Noura Farra, Sarah Alkuhlani, and Kemal Oflazer. 2014. Large Scale Arabic Error Annotation: Guidelines and Framework. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), Reykjavik, Iceland. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/bouamor-2015-qcmuq-qalb-2015-shared-task/