Elamur Research⌕
☰
home / papers / 2015 / TECHLIMED@QALB-Shared Task 2015: a hybrid Arabic Error Correction System

TECHLIMED@QALB-Shared Task 2015: a hybrid Arabic Error Correction System

Paper · 2015 · Djamel Mostefa, Jaber Abualasal, Omar Asbayou et al.

Summary

The authors describe Techlimed's entry in the native-speaker (L1) track of the QALB-2015 shared task. The task is automatic correction of Arabic text, using reader comments posted on Aljazeera articles. They built two systems. The first is a lexicon-based checker: Hunspell with an adapted Ayaspell Arabic dictionary and affix file. The second is a hybrid pipeline with three parts:

  • the morphological analyzer MADAMIRA,
  • a Moses phrase-based statistical MT model that "translates" analyzed text into corrected text,
  • a small set of deterministic clean-up rules.

The language model was trained on roughly 300M words of Arabic newspaper text. Hunspell performed poorly because it ignores punctuation, which accounts for about a third of the errors. The best hybrid variant, TECH-2, reached P 71.20 / R 64.94 / F1 67.93 on the blind test set. An ablation shows that each stage adds recall at a modest cost in precision. The paper also lists the error types that remain hard. Its main value is showing that analysis, statistical rewriting and simple rules work best together for a morphologically rich, noisy language.

Key points

  • Data and scoring. Alj-train-2014 has 20,428 sentences (1.1M tokens). The blind Alj-test-2015 set has 920 sentences (about 48k tokens). Scoring used the M2 scorer with Levenshtein-based edit alignment.

  • Hunspell baseline. On the dev set it scored P 56.64, R 19.78, F1 29.32. Its low recall is largely due to punctuation errors (~30% of all errors), which it cannot fix.

  • Hybrid setup. Moses was used with GIZA++ alignment and an IRSTLM language model. The LM corpora were OSAC (20M words), Adjir (147M) and crawled newspapers, about 300M words in total.

  • Variants.

    • TECH-2 trained on train+dev and was the best system.
    • TECH-3 used the raw erroneous text instead of MADAMIRA output as the SMT source and dropped to F1 64.85.
    • Combining Hunspell with SMT did not help.

  • TECH-2 ablation on the test set:

    StagePRF1
    MADAMIRA alone80.3339.9853.39
    +SMT70.8959.1264.89
    +rules71.2064.9467.93

Interesting ideas

  • I1. Using the analyzer's normalized output, rather than the raw noisy text, as the source side of the correction model made the rewriting task easier and beat the raw-source variant.
  • I2. A layered ablation (analyzer alone, then +SMT, then +rules) shows exactly where precision is traded for recall, which makes each stage's contribution explicit.
  • I3. A handful of cheap deterministic rules added about three F1 points on top of a statistical model. These normalize digit scripts, split digits from words, add a missing final stop and collapse repeated punctuation.
  • I4. The error analysis flags patterns that a pure dictionary checker misses: elongated letter repetitions, two words glued together, and homophone spellings that should both count as correct.
  • I5. Morphosyntactic context licenses fixes that a word list cannot make. One example is fixing a final letter only when the noun carries the definite article.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): self-induced repair rules could be mined from pairs of normalized tokens and gold text, not from raw input, so the rules generalize over surface noise.
  • I2 (learn): evaluation reports could add pipeline stages one at a time with P/R/F per stage, so we can see which stage buys coverage and which costs accuracy.
  • I3 (tokens): a transparent pre-normalization pass for web text, covering repeated "!!!", missing final periods and glued numbers, would steady the tokenizer and tagger before parsing.
  • I4 (tokens): explicit handlers for letter elongation and run-together words, plus accepted-variant lists, would cut spurious OOV lemmas in informal English input.
  • I5 (tokens): spelling or lemma repairs could be conditioned on tagger and morphology features, so every correction is explainable by a stated grammatical condition.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🔬Centre de Recherche en Linguistique AppliquéeLyon · France3 works · 24 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksEducation · 203 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Djamel Mostefa, Jaber Abualasal, Omar Asbayou, Mahmoud Gzawi and Ramzi Abbes. 2015. TECHLIMED@QALB-Shared Task 2015: a hybrid Arabic Error Correction System. Proceedings of the Second Workshop on Arabic Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w15-3220

@inproceedings{mostefa2015techlimed,
  title = {{TECHLIMED@QALB-Shared Task 2015: a hybrid Arabic Error Correction System}},
  author = {Djamel Mostefa, Jaber Abualasal, Omar Asbayou, Mahmoud Gzawi and Ramzi Abbes},
  booktitle = {Proceedings of the Second Workshop on Arabic Natural Language Processing},
  year = {2015},
  doi = {10.18653/v1/w15-3220},
  url = {https://doi.org/10.18653/v1/w15-3220}
}

References

17 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abbès, Ramzi Dichy, Joseph and Mohamed Hassoun. "The architecture of a standard arabic lexical database: some figures, ratios and categories from the Diinar. 1 source program." In Proceedings of the Workshop on Computational Approaches to Arabic Script-based Languages. Association for Computational Linguistics, 2004. 15–22. — The Architecture of a Standard Arabic Lexical Database. Some Figures, Ratios and Categories from the DIINAR.1 Source Program
  2. Abdelali, Ahmed. http://aracorpus.e3rab.com/. 2005. http://aracorpus.e3rab.com/ (accessed 2015). ↗
  3. Aboelezz, Mariam. "Latinised arabic and connections to bilingual ability." In Papers from the Lancaster University Postgraduate Conference in Linguistics and Language Teaching. 2009. find
  4. Ayaspell. Ayaspell Arabic dictionary project. 2008. http://ayaspell.sourceforge.net. ↗
  5. Buckwalter, Tim. Buckwalter Arabic Morphological Analyzer Version 1.0. 2002. http://catalog.ldc.upenn.edu/LDC2002L49 (accessed 06 2015). ↗
  6. Dahlmeier, Daniel and Ng, Hwee Tou. "Better evaluation for grammatical error correction." In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 2012. 568–572. find
  7. Hunspell. Hunspell. 2007. http://hunspell.sourceforge.net/ (accessed 2015). ↗
  8. Koehn, Philipp Hoang, Hieu Birch, Alexandra Callison-Burch, Chris Federico, Marcello Bertoldi, Nicola Cowan, Brooke Shen, Wade Moran, Christine Zens, Richard Dyer, Christopher Bojar, Ondrej Constantin, Alexandra and Herbst. Evan. "Open source toolkit for statistical machine translation." 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions. Association for Computational Linguistics, 2007. 177-180. find
  9. Levenshtein, Vladimir. "Binary codes capable of correcting deletions, insertions and reversals." In Soviet physics doklady. 1966. volume 10, page 707. find
  10. Mohit, Behrang Rozovskaya, Alla Habash, Nizar Zaghouani, Wajd and Obeid, Ossama. "The First QALB Shared Task on Automatic Text Correction for Arabic." Proceedings of EMNLP Workshop on Arabic Natural Language Processing. Doha, Qatar, 2014.
  11. Mostefa, Djamel Asbayou, Omar and Abbes, Ramzi. "TECHLIMED System Description for the Shared Task on Automatic Arabic Error Correction." Proceedings of EMNLP Workshop on Arabic Natural Language Processing. Doha, Qatar, 2014.
  12. Och, Franz Joseph and Ney, Hermann. "A systematic comparison of various statistical alignment models." Computational Linguistics. 2003. 29(1):19–51. ↗
  13. Pasha, Arfath Al-Badrashiny, Mohamed El Kholy, Ahmed Eskander, Ramy Diab, Mona Habash, Nizar Pooleery, Manoj Rambow, Owen and Roth, Ryan. "MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic." Proceedings of LREC'2014.
  14. Rozovskaya, Alla Bouamor, Houda Habash, Nizar Zaghouani, Wajdi Obeid, Ossama and Mohit, Behrang. "The Second QALB Shared Task on Automatic Text Correction for Arabic." Proceedings of ACL Workshop on Arabic Natural Language. Beijing, China, 2015.
  15. Saad, Ashour and Motaz, Wesam. "Osac: Open source arabic corpora." In 6th ArchEng Int. Symposiums, EEECS. 2010. volume 10. ↗
  16. Zaghouani, Wajdi Habash, Nizar Bouamor, Houda Rozovskaya, Alla Mohit, Behrang Heider, Abeer and Oflazer, Kemal. "Correction Annotation for Non-Native Arabic Texts: Guidelines and Corpus." Proceedings of The 9th Linguistic Annotation Workshop. Denver, Colorado, USA: Association for Computational Linguistics, 2015. 129-139.
  17. Zaghouani, Wajdi Mohit, Behrang Habash, Nizar Obeid, Ossama Tomeh, Nadi Rozovskaya, Alla Farra, Noura Alkuhlani, Sarah and Oflazer, Kemal. "Large scale arabic error annotation: Guidelines and framework." In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14). Reykjavik, Iceland: European Language Resources Association (ELRA), 2014. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mostefa-2015-techlimed-qalb-shared-task-2015/