Elamur Research⌕
☰
home / papers / 2019 / Segmentation for Domain Adaptation in Arabic

Segmentation for Domain Adaptation in Arabic

Paper · 2019 · Mohammed Attia and Ali Elkahky

Summary

Arabic NLP models trained on edited Modern Standard Arabic (MSA) perform poorly on dialectal, user-generated text. Much of the vocabulary is actually shared, but heavy cliticization, gemination and letter repetition hide the overlap. The authors argue that splitting clitics and normalizing word bases can expose this common lexicon and cut data sparsity. They define a segmentation convention in which every clitic that can bear its own POS tag and occupy a node in a dependency tree becomes a separate token, and they annotate a small Egyptian (EG) dataset under it. Adding this data to MSA segmentation training sharply improves Egyptian segmentation. Even with no dialectal POS data at all, it lifts Egyptian POS tagging by 7.37 points absolute, mainly because fewer tokens are unknown to the MSA-trained tagger. The work shows that cheap, low-level annotation in a new domain can propagate gains up a pipeline. From this the authors propose a "data trapezoid": plentiful segmentation data at the base, less POS data above it, and the least treebank data at the top.

Key points

  • Segmentation F1, MSA-only model: 97.91 on MSA test and 82.56 on EG test.

  • Segmentation F1, MSA+EG model: 97.62 on MSA and 91.40 on EG. That is +8.84 on EG for a loss of about 0.3 on MSA, and it is close to earlier Egyptian results (91.90 by Mohamed et al. 2012; 92.65 by Samih et al. 2017a).

  • POS F1 on EG, MSA-trained tagger (MSA scores in brackets):

    Segmentation inputEGMSA
    MSA segmenter66.7094.36
    MSA+EG segmenter74.0794.10
    Gold segmentation81.3396.66

  • OOV reduction: the MSA+EG segmenter cuts OOV tokens for the tagger by 5 points absolute, about 25% relative.

  • Convention choices:

    • The definite article Al is split off as a clitic, unlike the ATB, where it is a definiteness marker.
    • Unlike Farasa, affixes such as the feminine marker are not split.
    • Hashtags, emoticons and user names stay whole.
    • Misspelled words are segmented as if they were spelled correctly.

Interesting ideas

  • I1. Decide token boundaries by a syntactic test: a piece is split off only if it gets its own POS tag and tree node, while inflectional affixes stay attached. This gives tokenization a principled link to the parser.
  • I2. Feed a downstream model three kinds of input (baseline predictions, improved predictions, gold) to separate what has been gained from the headroom still available.
  • I3. Measure the OOV rate of the downstream model under each upstream variant, to give a mechanistic reason why an upstream change helps.
  • I4. Segment noisy words as if they were spelled correctly, keeping boundaries despite repeated letters. Normalization then serves structure rather than replacing the surface form.
  • I5. Allocate annotation as a "data trapezoid": the most data for the cheapest low layer, less for POS, and the least for dependency trees, matching cost to layer.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): Apply the same criterion to English contractions and possessive 's, so each split piece maps to exactly one UD node and the choice can be explained.
  • I2 (learn): Run the tagger and parser on gold tokenization alongside predicted tokenization, to show how much of the parse error comes from the tokens stage.
  • I3 (learn): Report OOV counts against the lexicon and the gparse vocabulary for each tokenizer or normalizer version, as a cheap diagnostic alongside accuracy.
  • I4 (tokens): Map elongated or misspelled informal English ("sooooo", "gonna") to canonical tokens and lemmas, keeping the original form for explainability.
  • I5 (learn): Budget silver and LLM-teacher data per layer, with large volumes for tokens and tags and fewer but higher-confidence tree annotations, when adapting to a new genre such as tales.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Mohammed Attia and Ali Elkahky. 2019. Segmentation for Domain Adaptation in Arabic. Proceedings of the Fourth Arabic Natural Language Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w19-4613

@inproceedings{attia2019segmentation,
  title = {{Segmentation for Domain Adaptation in Arabic}},
  author = {Mohammed Attia and Ali Elkahky},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Workshop},
  year = {2019},
  doi = {10.18653/v1/w19-4613},
  url = {https://doi.org/10.18653/v1/w19-4613}
}

References

27 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed Abdelali, Kareem Darwish, Nadir Durrani, and Hamdy Mubarak. 2016. Farasa: A fast and furious segmenter for arabic. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 11–16.
  2. Nizar Habash, Owen Rambow, and Ryan Roth. 2009. MADA+TOKAN: A toolkit for arabic tokenization, diacritization, morphological disambiguation, pos tagging, stemming and lemmatization. In Proceedings of the 2nd international conference on Arabic language resources and tools (MEDAR), Cairo, Egypt, volume 41, page 62. find
  3. Ahmed H. Aliwy. 2012. Tokenization as preprocessing for arabic tagging system. International Journal of Information and Education Technology, 2(4):348. ↗
  4. Nizar Habash, Ryan Roth, Owen Rambow, Ramy Eskander, and Nadi Tomeh. 2013. Morphological analysis and disambiguation for dialectal arabic. In Hlt-Naacl, pages 426–432.
  5. Mohammed Attia, Pavel Pecina, Younes Samih, Khaled Shaalan, and Josef Van Genabith. 2012. Improved spelling error detection and correction for arabic. Proceedings of COLING 2012: Posters, pages 103–112.
  6. Jan Hajič, Barbora Vidová-Hladká, and Petr Pajas. 2001. The prague dependency treebank: Annotation structure and support. In Proceedings of the IRCS Workshop on Linguistic Databases, pages 105–114. find
  7. Djallel Bouneffouf, Romain Laroche, Tanguy Urvoy, Raphael Féraud, and Robin Allesiardo. 2014. Contextual bandit for active learning: Active thompson sampling. In International Conference on Neural Information Processing, pages 405–412. Springer. find
  8. Vladimir I. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physics doklady, volume 10, pages 707–710. find
  9. Danqi Chen and Christopher Manning. 2014. A fast and accurate dependency parser using neural networks. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 740–750.
  10. Mohamed Maamouri, Ann Bies, Tim Buckwalter, and Wigdan Mekki. 2004. The penn arabic treebank: Building a large-scale annotated arabic corpus. In NEMLAR conference on Arabic language resources and tools, volume 27, pages 466–467. Cairo. find
  11. Marie-Catherine De Marneffe and Christopher D. Manning. 2008. The stanford typed dependencies representation. In COLING 2008: Proceedings of the Workshop on Cross-framework and Cross-domain Parser Evaluation, pages 1–8. Association for Computational Linguistics.
  12. Ryan McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage, Yoav Goldberg, Dipanjan Das, Kuzman Ganchev, Keith Hall, Slav Petrov, Hao Zhang, Oscar Täckström, Claudia Bedini, Núria Bertomeu Castelló, and Jungmee Lee. 2013. Universal dependency annotation for multilingual parsing. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 92–97, Sofia, Bulgaria. Association for Computational Linguistics.
  13. Mona Diab. 2009. Second generation amira tools for arabic processing: Fast and robust tokenization, pos tagging, and base phrase chunking. In 2nd International Conference on Arabic Language Resources and Tools, volume 110. find
  14. Mona Diab, Kadri Hacioglu, and Daniel Jurafsky. 2004. Automatic tagging of arabic text: From raw text to base phrase chunks. In Proceedings of HLT-NAACL 2004: Short papers, pages 149–152. Association for Computational Linguistics.
  15. Emad Mohamed. 2018. Morphological segmentation and part-of-speech tagging for the arabic heritage. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 17(3):22. ↗
  16. Emad Mohamed, Behrang Mohit, and Kemal Oflazer. 2012. Annotating and learning morphological segmentation of egyptian colloquial arabic. In LREC, pages 873–877.
  17. Mohamed Eldesouki, Younes Samih, Ahmed Abdelali, Mohammed Attia, Hamdy Mubarak, Kareem Darwish, and Kallmeyer Laura. 2017. Arabic multi-dialect segmentation: bi-lstm-crf vs. svm. arXiv preprint arXiv:1708.05891. ↗
  18. Will Monroe, Spence Green, and Christopher D. Manning. 2014. Word segmentation of informal arabic with domain adaptation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 206–211.
  19. Nizar Habash, Ramy Eskander, and Abdelati Hawwari. 2012. A morphological analyzer for egyptian arabic. In Proceedings of the twelfth meeting of the special interest group on computational morphology and phonology, pages 1–9. Association for Computational Linguistics.
  20. Joakim Nivre. 2003. An efficient algorithm for projective dependency parsing. In Proceedings of the 8th International Workshop on Parsing Technologies (IWPT).
  21. Nizar Habash and Owen Rambow. 2005. Arabic tokenization, part-of-speech tagging and morphological disambiguation in one fell swoop. In Proceedings of Joakim Nivre, Johan Hall, and Jens Nilsson. 2004. Memory-based dependency parsing. In Proceedings of the Eighth Conference on Computational Natural Language Learning (CoNLL-2004) at HLT-NAACL 2004.
  22. Joakim Nivre, Johan Hall, Jens Nilsson, Atanas Chanev, Gülşen Eryigit, Sandra Kübler, Svetoslav Marinov, and Erwin Marsi. 2007. Maltparser: A language-independent system for data-driven dependency parsing. Natural Language Engineering, 13(2):95–135. ↗
  23. Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D. Manning, Ryan T. McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, et al. 2016. Universal dependencies v1: A multilingual treebank collection. In International Conference on Language Resources and Evaluation (LREC 2016), pages 1659– 1666, Portorož, Slovenia.
  24. Younes Samih, Mohammed Attia, Mohamed Eldesouki, Ahmed Abdelali, Hamdy Mubarak, Laura Kallmeyer, and Kareem Darwish. 2017a. A neural architecture for dialectal arabic segmentation. In Proceedings of the Third Arabic Natural Language Processing Workshop, pages 46–54.
  25. Younes Samih, Mohamed Eldesouki, Mohammed Attia, Kareem Darwish, Ahmed Abdelali, Hamdy Mubarak, and Laura Kallmeyer. 2017b. Learning from relatives: unified dialectal arabic segmentation. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 432–441.
  26. Robert A. Wagner and Michael J. Fischer. 1974. The string-to-string correction problem. Journal of the ACM (JACM), 21(1):168–173. ↗
  27. Hiroyasu Yamada and Yuji Matsumoto. 2003. Statistical dependency analysis with support vector machines. In Proceedings of the 8th International Workshop on Parsing Technologies, pages 195–206, Nancy, France.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/attia-2019-segmentation-domain-adaptation-arabic/