Elamur Research⌕
☰
home / papers / 2014 / Arabic Native Language Identification

Arabic Native Language Identification

Paper · 2014 · Shervin Malmasi and Mark Dras

Summary

Native Language Identification (NLI) predicts an author's first language (L1) from their writing in a second language. Before this work it had been studied almost only on English. Malmasi and Dras run the first NLI experiment on learners of Arabic, a right-to-left language with rich morphology. They use the second version of the Arabic Learner Corpus (ALC) and keep the seven L1 groups that have the most texts, 329 essays in total.

The pipeline has four steps:

  • Texts are segmented with the Stanford Word Segmenter.
  • They are tagged and parsed with the Stanford Arabic Parser.
  • They are classified with a linear SVM (LIBLINEAR).
  • Results come from 10-fold cross-validation.

Only syntactic features are used: unlexicalized CFG production rules, function-word frequencies and POS n-grams. Word features are left out because topics are unevenly spread across L1 groups. Combining all features gives 41% accuracy, against a 23.1% majority baseline. This is the first classifier-based evidence of L1 transfer in learner Arabic. The authors note uses in SLA research, L1-aware teaching materials and forensic authorship profiling.

Key points

  • Data: ALC v2 subset with seven L1s and 329 texts.
    • Chinese 76, Urdu 64, Malay 46, French 44, Fulani 36, English 35, Yoruba 28.
    • L1s with fewer than about 10 texts were dropped.
    • Essays by native Arabic speakers were excluded.
  • Single-feature accuracy:
    • CFG production rules: 31.7%.
    • 150 Arabic function words: 29.2%.
    • POS unigrams / bigrams / trigrams: 36.0 / 37.6 / 36.5%.
  • Combined result: merging all feature types into one space gave the best score, 41.0%, which suggests the feature types carry different information.
  • Setup:
    • All features are relative frequencies.
    • The classifier is a LIBLINEAR linear SVM, evaluated with 10-fold cross-validation, the usual NLI reporting standard.
    • The baseline is the majority class: 76/329 = 23.1%.
  • Main limitation: the authors call this the smallest NLI dataset in their knowledge. Yet single models score only about 10 points below a comparable 7-class English study that had far more data. They expect accuracy to rise as larger Arabic learner corpora appear, such as the Qatar Arabic Language Bank.

Interesting ideas

  • I1. Lexical n-grams were dropped on purpose because topic was unevenly spread across classes, so a classifier could have learned topic instead of L1. This is a concrete way to avoid a confounding variable.
  • I2. Unlexicalized production rules taken from parse trees act as a fingerprint of the global syntactic habits of a group of writers.
  • I3. Frequencies of a fixed list of function words give a cheap, topic-independent signal of grammatical style.
  • I4. Each feature family is scored on its own and then all are merged. A gain from merging shows the families are complementary, not redundant. Results are always compared against a majority-class baseline.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): when testing whether a learned rule or classifier captures structure, add a negative control. Retrain or evaluate with lexical cues removed, or with topic shuffled, so lexical shortcuts cannot explain the gain.
  • I2 (tree): frequency profiles of unlexicalized dependency or constituency patterns could flag which treebank dialect, or which witness parser, produced a tree. Unusual profiles could also point to systematic parser drift.
  • I3 (frames): function-word profiles could be an explainable cue for register or speech-act style, such as narration versus instruction in fables. This would not rely on content words.
  • I4 (learn): when adding a new witness parser or feature source, report its solo score, the merged score and the majority baseline. That makes it clear whether it brings new information.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Macquarie UniversitySydney · Australia42 works · 5,776 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksSocial Psychology · 306 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Shervin Malmasi and Mark Dras. 2014. Arabic Native Language Identification. Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/w14-3625

@inproceedings{malmasi2014arabic,
  title = {{Arabic Native Language Identification}},
  author = {Shervin Malmasi and Mark Dras},
  booktitle = {Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP)},
  year = {2014},
  doi = {10.3115/v1/w14-3625},
  url = {https://doi.org/10.3115/v1/w14-3625}
}

References

29 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed Abbasi and Hsinchun Chen. 2005. Applying authorship analysis to extremist-group Web forum messages. IEEE Intelligent Systems, 20(5):67–75. ↗
  2. Ghazi Abuhakema, Reem Faraj, Anna Feldman, and Eileen Fitzpatrick. 2008. Annotating an Arabic Learner Corpus for Error. In LREC. find
  3. Abdullah Alfaifi and Eric Atwell. 2013. Arabic Learner Corpus v1: A New Resource for Arabic Language Research. find
  4. Abdullah Alfaifi, Eric Atwell, and I Hedaya. 2014. Arabic learner corpus (ALC) v2: a new written and spoken corpus of Arabic learners. In Proceedings of the Learner Corpus Studies in Asia and the World (LCSAW), Kobe, Japan. find
  5. Julian Brooke and Graeme Hirst. 2011. Native language detection with ‘cheap’ learner corpora. In Conference of Learner Corpus Research (LCR2011), Louvain-la-Neuve, Belgium. Presses universitaires de Louvain. find
  6. Malcolm Coulthard and Alison Johnson. 2007. An introduction to Forensic Linguistics: Language in evidence. Routledge. ↗
  7. Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. LIBLINEAR: A library for large linear classification. Journal of Machine Learning Research, 9:1871–1874. ↗
  8. John Gibbons and Venn Prakasam. 2004. Language in the Law. Orient Blackswan. find
  9. John Gibbons. 2003. Forensic Linguistics: An Introduction To Language In The Justice System. find
  10. Sylviane Granger. 2012. Learner corpora. The Encyclopedia of Applied Linguistics. find
  11. Tim Grant. 2007. Quantifying evidence in forensic authorship analysis. International Journal of Speech Language and the Law, 14(1):1–25. ↗
  12. Nizar Y Habash. 2010. Introduction to Arabic natural language processing. Synthesis Lectures on Human Language Technologies, 3(1):1–187. ↗
  13. Batia Laufer and Nany Girsai. 2008. Form-focused instruction in second language vocabulary learning: A case for contrastive analysis and translation. Applied Linguistics, 29(4):694–716. ↗
  14. Shervin Malmasi and Mark Dras. 2014a. Chinese Native Language Identification. Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics. find
  15. Shervin Malmasi and Mark Dras. 2014b. From Visualisation to Hypothesis Construction for Second Language Acquisition. In Graph-Based Methods for Natural Language Processing, Doha, Qatar, October. Association for Computational Linguistics. find
  16. Shervin Malmasi and Mark Dras. 2014c. Language Transfer Hypotheses with Linear SVM Weights. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). find
  17. Shervin Malmasi, Sze-Meng Jojo Wong, and Mark Dras. 2013. Nli shared task 2013: Mq submission. In Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications, pages 124–133, Atlanta, Georgia, June. Association for Computational Linguistics. find
  18. Fethi Mansouri. 2005. Agreement morphology in Arabic as a second language. Cross-linguistic aspects of Processability Theory, pages 117–253. find
  19. Gerald R McMenamin. 2002. Forensic linguistics: Advances in Forensic Stylistics. CRC press. find
  20. Behrang Mohit. 2013. QALB: Qatar Arabic language bank. In Qatar Foundation Annual Research Conference, number 2013. find
  21. Helle Lykke Nielsen. 1997. On acquisition order of agreement procedures in Arabic learner language. Al-Arabiyya, 30:49–93. find
  22. Ria Perkins. 2014. Linguistic identifiers of L1 Persian speakers writing in English: NLID for authorship analysis. Ph.D. thesis, Aston University. ↗
  23. Karin C. Ryding. 2013. Teaching Arabic in the United States. In Kassem M Wahba, Zeinab A Taha, and Liz England, editors, Handbook for Arabic language teaching professionals in the 21st century. Routledge. find
  24. Ben Swanson and Eugene Charniak. 2014. Data Driven Language Transfer Hypotheses. EACL 2014, page 169. find
  25. Joel Tetreault, Daniel Blanchard, Aoife Cahill, Beata Beigman-Klebanov, and Martin Chodorow. 2012. Native Tongues, Lost and Found: Resources and Empirical Evaluations in Native Language Identification. In Proc. Internat. Conf. on Computat. Linguistics (COLING). find
  26. Joel Tetreault, Daniel Blanchard, and Aoife Cahill. 2013. A report on the first native language identification shared task. In Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications, pages 48–57, Atlanta, Georgia, June. Association for Computational Linguistics. find
  27. Kassem M Wahba, Zeinab A Taha, and Liz England. 2013. Handbook for Arabic language teaching professionals in the 21st century. Routledge. ↗
  28. Sze-Meng Jojo Wong and Mark Dras. 2009. Contrastive analysis and native language identification. In Proc. Australasian Language Technology Workshop (ALTA), pages 53–61. find
  29. Wajdi Zaghouani, Behrang Mohit, Nizar Habash, Ossama Obeid, Nadi Tomeh, Alla Rozovskaya, Noura Farra, Sarah Alkuhlani, and Kemal Oflazer. 2014. Large Scale Arabic Error Annotation: Guidelines and Framework. In Nicoletta Calzolari (Conference Chair), Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, and Stelios Piperidis, editors, Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), Reykjavik, Iceland, may. European Language Resources Association (ELRA). ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/malmasi-2014-arabic-native-language-identification/