Elamur Research⌕
☰
home / papers / 2018 / Studying the History of the Arabic Language: Language Technology and a Large-Scale Historical Corpus

Studying the History of the Arabic Language: Language Technology and a Large-Scale Historical Corpus

Paper · 2018 · Yonatan Belinkov, Alexander Magidow, Alberto Barron-Cedeno et al.

Summary

Written Arabic was standardized very early and is often described as almost unchanged for over a millennium. Because of this, its history has mostly been studied by hand, on small samples and a few selected features. The authors build data and tools to test this view at scale. They take OpenITI, a diachronic corpus of about 1.5 G words covering roughly 1400 years. They run Arabic NLP tools over it: Farasa-based segmentation, part-of-speech tags, lemmas and syntactic parse trees. They also adapt a text reuse detector, originally built for a smaller Hebrew/Aramaic corpus, to find quoted and paraphrased passages so these can be removed. On the cleaned corpus they apply a new periodization algorithm based on word embeddings. It compares language use across whole sub-corpora rather than a handful of hand-picked features. The results confirm the split between Classical Arabic and Modern Standard Arabic and suggest further breaks inside the classical period. The processed corpus and code are released, so others can study Arabic, or other languages, with data-driven methods.

Key points

  • OpenITI has about 1.5 G words and is described as the largest publicly available historical Arabic corpus. It was released with segmentation, POS tags, lemmas and parse trees.
  • The adapted reuse detector found and removed 292 M words of reused text, close to 20% of the corpus. The result is a "hollowed" version in both plain-text and preprocessed form.
  • Word lifespans confirm that Arabic vocabulary changes more slowly than English vocabulary.
  • Automatic periodization separates the oldest and the most modern texts from a core Classical Arabic block. This block runs from about 400 AH (1009 CE) to 1300 AH (1882 CE).
  • The study also finds weaker evidence for:
    • an early/late Classical break around the 1000s CE, earlier than usually assumed;
    • a possible new division in the late 15th century CE.
  • Checking specific claims from the literature gave mixed results:
    • The adverb AyDA ('also') appears as early as a text dated 68 AH (688 CE), long before its claimed Post-Classical onset.
    • Suffixes such as -Any and -yp do rise, but already before 1258 CE.

Interesting ideas

  • I1. Detecting and removing reused or quoted text before computing statistics stops material from one era from contaminating measurements of another; it is a general guard against duplicated evidence.
  • I2. The authors test scholarly claims directly against corpus frequencies, and they first strip clitics with a segmenter so that counts reflect the real word form rather than its attached variants.
  • I3. They deliberately drop a feature (xASp 'especially') because it is a homograph of another word ('the elite') and cannot be told apart automatically. Not measuring is better than measuring the wrong thing.
  • I4. Periodization compares embedding-level language use across whole sub-corpora instead of a few hand-chosen features, so data boundaries emerge without prior assumptions.
  • I5. Texts are dated by the author's date of death, which gives only an upper bound. The authors reason carefully about what such bounds allow them to conclude (for example, that a trend began before 1258).

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Remove near-duplicate passages between silver training data and evaluation sets so that agreement scores among witness parsers are not inflated by memorized text.
  • I2 (tokens): Normalize clitics and inflection to lemmas before counting, so that frequency-based morphology rules rest on clean counts and can be checked against claims found in grammars.
  • I3 (tokens): Mark ambiguous forms that the tagger cannot resolve and leave them out of downstream statistics or explanations, instead of silently guessing.
  • I4 (tree): Compare treebanks or text domains the same way to find where annotation dialects or genres actually diverge, which can guide UD dialect conversion.
  • I5 (world): When building event order, store bounded or approximate time anchors and reason about what they permit, rather than forcing exact dates.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Massachusetts Institute of TechnologyCambridge · United States211 works · 69,000 citations🎓Bar-Ilan UniversityRamat Gan · Israel141 works · 23,225 citations🎓Harvard UniversityCambridge · United States112 works · 42,023 citations🎓Hamad bin Khalifa UniversityDoha · Qatar141 works · 14,175 citations🔬MIT Computer Science and Artificial Intelligence LaboratoryCambridge · United States69 works · 30,953 citations🎓University of ViennaVienna · Austria16 works · 38,729 citations🎓University of Rhode IslandKingston · United States2 works · 99 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksLiterature and Literary Theory · 519 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Yonatan Belinkov, Alexander Magidow, Alberto Barron-Cedeno, Avi Shmidman and Maxim Romanov. 2018. Studying the History of the Arabic Language: Language Technology and a Large-Scale Historical Corpus. arXiv:1809.03891.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1809.03891 · arXiv 1809.03891

@misc{belinkov2018studying,
  title = {{Studying the History of the Arabic Language: Language Technology and a Large-Scale Historical Corpus}},
  author = {Yonatan Belinkov, Alexander Magidow, Alberto Barron-Cedeno, Avi Shmidman and Maxim Romanov},
  year = {2018},
  doi = {10.48550/arxiv.1809.03891},
  eprint = {1809.03891},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1809.03891}
}

References

71 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdelali, A., Darwish, K., Durrani, N., Mubarak, H.: Farasa: A Fast and Furious Segmenter for Arabic. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pp. 11–16. Association for Computational Linguistics (2016). DOI 10.18653/v1/N16-3003. URL http://aclanthology.coli.uni-saarland.de/pdf/N/N16/N16-3003.pdf ↗
  2. Al-Jallad, A.: An outline of the grammar of the Safaitic inscriptions. No. 80 in Studies in Semitic languages and linguistics. Brill (2015) find
  3. Al-Sulaiti, L.: Designing and Developing a Corpus of Contemporary Arabic. Master’s thesis, The University of Leeds, Leeds, UK (2004) find
  4. Al-Thubaity, A.O.: A 700M+ Arabic Corpus: KACST Arabic Corpus Design and Construction. Lang. Resour. Eval. 49(3), 721–751 (2015) find
  5. Ali, A.S.M.: A linguistic study of the development of scientific vocabulary in Standard Arabic. Keegan Paul International (1987) find
  6. Alrabiah, M., Al-Salman, A., Atwell, E.: The design and construction of the 50 million words KSUCCA. In: Proceedings of WACL2 Second Workshop on Arabic Corpus Linguistics, pp. 5–8 (2013) find
  7. Arts, T., Belinkov, Y., Habash, N., Kilgarriff, A., Suchomel, V.: arTenTen: Arabic Corpus and Word Sketches. Journal of King Saud University - Computer and Information Sciences 26(4), 357 – 371 (2014). Special Issue on Arabic NLP find
  8. Belinkov, Y., Magidow, A., Romanov, M., Shmidman, A., Koppel, M.: Shamela: A large-scale historical arabic corpus. In: Proceedings of the Workshop on Language Technology Resources and Tools for Digital Humanities (LT4DH at Coling), pp. 45–53. The COLING 2016 Organizing Committee, Osaka, Japan (2016)
  9. Bensalem, I., Boukhalfa, I., Rosso, P., Lahsen, A., Darwish, K., Chikhi, S.: Overview of the AraPlagDet PAN@ FIRE2015 Shared Task on Arabic Plagiarism Detection. In: Notebook Papers of FIRE 2015 (CEUR-WS vol. 1587), pp. 111–122. Gandhinagar, India (2015) find
  10. Björkelund, A., Çetinoğlu, Ö., Farkas, R., Mueller, T., Seeker, W.: (Re) Ranking Meets Morphosyntax: State-of-the-Art Results from the SPMRL 2013 Shared Task pp. 135–145 (2013) find
  11. Braschler, M., Harman, D. (eds.): Notebook Papers of CLEF 2010 LABs and Workshops. Padua, Italy (2010) find
  12. Chambers, N.: Labeling Documents with Timestamps: Learning from their Time Expressions. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 98–106. Jeju Island, Korea (2012) find
  13. Claridge, C.: Historical corpora. Corpus linguistics: An international handbook 1, 242–259 (2008) find
  14. Clough, P., Gaizauskas, R.: Corpora and Text Re-Use. In: A. Lüdeling, M. Kytö, T. McEnery (eds.) Handbook of Corpus Linguistics, Handbooks of Linguistics and Communication Science, pp. 1249—1271. Mouton de Gruyter (2009) find
  15. Clough, P., Gaizauskas, R., Piao, S., Wilks, Y.: Measuring Text Reuse. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL 2002), pp. 152–159. Association for Computational Linguistics, Philadelphia, PA (2002) find
  16. Dalli, A., Wilks, Y.: Automatic Dating of Documents and Temporal Text Classification. In: Proceedings of the Workshop on Annotating and Reasoning about Time and Events, pp. 17–22. Sydney, Australia (2006) find
  17. Darwish, K., Abdelali, A., Mubarak, H.: Using Stem-Templates to Improve Arabic POS and Gender/Number Tagging. In: Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), LREC 2014, pp. 2926–2931. European Language Resources Association (ELRA) (2014). URL http://www.lrec-conf.org/proceedings/lrec2014/pdf/335_Paper.pdf ↗
  18. Darwish, K., Mubarak, H.: Farasa: A New Fast and Accurate Arabic Word Segmenter. In: LREC (2016)
  19. Darwish, K., Mubarak, H., Abdelali, A., Eldesouki, M.: Arabic POS Tagging: Don’t Abandon Feature Engineering Just Yet. In: Proceedings of the Third Arabic Natural Language Processing Workshop, pp. 130–137. Association for Computational Linguistics (2017). URL http://aclweb.org/anthology/W17-1316 ↗
  20. Davies, M.: The Corpus of Historical American English: 400 million words, 1810-2009. http://corpus.byu.edu/coha (2010) ↗
  21. Dubossarsky, H., Tsvetkov, Y., Dyer, C., Grossman, E.: A bottom up approach to category mapping and meaning change. In: V. Pirrelli, C. Marzi, M. Ferro (eds.) Word Structure and Word Usage. Proceedings of the NetWordS Final Conference (2015) find
  22. Elewa, A.H.: Collocation and Synonymy in Classical Arabic: A Corpus-based Approach. Ph.D. thesis, The University of Manchester, Manchester, UK (2004) find
  23. Ferrando, I.: History of Arabic. In: K. Versteegh (ed.) Encyclopedia of Arabic Language and Linguistics, vol. 2, pp. 604–611. Brill, Leiden (2007) find
  24. Fischer, W.: Classical Arabic. In: K. Versteegh (ed.) Encyclopedia of Arabic Language and Lingusitics, vol. 1, pp. 397–405. Brill, Leiden (2006) find
  25. Gries, S.T., Hilpert, M.: Variability-based Neighbor Clustering: A bottom-up approach to periodization in historical linguistics. In: T. Nevalainen, E.C. Traugott (eds.) The Oxford Handbook of the History of English, pp. 134–144. Oxford University Press, Oxford (2012) find
  26. Grozea, C., Popescu, M.: The Encoplot Similarity Measure for Automatic Detection of Plagiarism - Notebook for PAN at CLEF 2011. In: V. Petras, P. Forner, P. Clough (eds.) Notebook Papers of CLEF 2011 LABs and Workshops. Amsterdam, The Netherlands (2011) find
  27. Habash, N., Rambow, O.: Arabic Tokenization, Part-of-Speech Tagging and Morphological Disambiguation in One Fell Swoop. In: Proceedings of ACL (2005)
  28. Habash, N., Rambow, O., Roth, R.: MADA+TOKAN: A Toolkit for Arabic Tokenization, Diacritization, Morphological Disambiguation, POS Tagging, Stemming and Lemmatization. In: Proceedings of the Second International Conference on Arabic Language Resources and Tools (2009) find
  29. Hall, D.L.W., Durrett, G., Klein, D.: Less Grammar, More Features. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pp. 228–237 (2014) find
  30. Hamilton, W.L., Leskovec, J., Jurafsky, D.: Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1489–1501. Association for Computational Linguistics (2016). DOI 10.18653/v1/P16-1141. URL http://aclanthology.coli.uni-saarland.de/pdf/P/P16/P16-1141.pdf ↗
  31. Hammo, B., Yagi, S., Ismail, O., AbuShariah, M.: Exploring and exploiting a historical corpus for Arabic. Language Resources and Evaluation 50(4), 839–861 (2016). DOI 10.1007/s10579-015-9304-9. URL https://doi.org/10.1007/s10579-015-9304-9 ↗
  32. Holes, C.: Modern Arabic: Structures, Functions, and Varieties. Georgetown University Press, Washington, D.C. (2004) find
  33. Ji, M.: A corpus-based study of lexical periodization in historical Chinese. Literary and Linguistic Computing 25(2), 199–213 (2010) find
  34. Joachims, T.: Training Linear SVMs in Linear Time. KDD ’06, pp. 217–226. ACM, New York, NY (2006) find
  35. de Jong, F., Rode, H., Hiemstra, D.: Temporal language models for the disclosure of historical text. Royal Netherlands Academy of Arts and Sciences (2005) find
  36. Kasprzak, J., Brandejs, M.: Improving the Reliability of the Plagiarism Detection System. Lab Report for PAN at CLEF 2010. In: Braschler and Harman [12] find
  37. Kilgarriff, A., Rychly, P., Smrz, P., Tugwell, D.: The Sketch Engine. In: Proceedings of EURALEX (2004) find
  38. Kim, Y., Chiu, Y.I., Hanaki, K., Hegde, D., Petrov, S.: Temporal Analysis of Language through Neural Language Models. In: Proceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Science, pp. 61–65. Association for Computational Linguistics, Baltimore, MD, USA (2014). URL http://www.aclweb.org/anthology/W14-2517 ↗
  39. Kulkarni, V., Al-Rfou, R., Perozzi, B., Skiena, S.: Statistically Significant Detection of Linguistic Change. In: Proceedings of the 24th International World Wide Web Conference, WWW ’15 (2015) find
  40. Lane, E.W.: Arabic-English Lexicon. Willams & Norgate (1863) find
  41. Lentin, J.: Middle Arabic. In: K. Versteegh (ed.) Encyclopedia of Arabic Language and Linguistics, vol. 1, online edition edn., pp. 87–96. Brill, Leiden (2006) find
  42. Li, W.P.: Language Technologies for Understanding Law, Politics, and Public Policy. Ph.D. thesis, Massachusetts Institute of Technology, Cambridge, MA, USA (2016) find
  43. Magidow, A.: A Digital Philological Investigation of the History of hā hunā Constructions. Romano-Arabica 16, 239–256 (2016) find
  44. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient Estimation of Word Representations in Vector Space. CoRR abs/1301.3781 (2013). URL http://arxiv.org/abs/1301.3781 ↗
  45. Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed Representations of Words and Phrases and their Compositionality. CoRR abs/1310.4546 (2013). URL http://arxiv.org/abs/1310.4546 ↗
  46. Mikolov, T., tau Yih, W., Zweig, G.: Linguistic Regularities in Continuous Space Word Representations. In: Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT-2013). Association for Computational Linguistics (2013). URL http://research.microsoft.com/apps/pubs/default.aspx?id=189726 ↗
  47. Mikolov, T., Yih, W., Zweig, G.: Linguistic Regularities in Continuous Space Word Representations. In: Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, June 9-14, 2013, Westin Peachtree Plaza Hotel, Atlanta, Georgia, USA, pp. 746–751 (2013). URL http://aclweb.org/anthology/N/N13/N13-1090.pdf ↗
  48. Newman, D.L.: The Arabic Literary Language: the nahd.a and beyond. In: J. Owens (ed.) The Oxford Handbook of Arabic Linguistics, pp. 472–494. Oxford University Press, Oxford (2013) find
  49. Niculae, V., Zampieri, M., Dinu, L., Ciobanu, A.M.: Temporal Text Ranking and Automatic Dating of Texts. In: Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, volume 2: Short Papers, pp. 17–21. Gothenburg, Sweden (2014). URL http://www.aclweb.org/anthology/E14-4004 ↗
  50. Osama Hamed, T.Z.: A Survey and Comparative Study of Arabic Diacritization Tools. JLCL 32(1), 27–47 (2017) find
  51. Pasha, A., Al-Badrashiny, M., Diab, M., Kholy, A.E., Eskander, R., Habash, N., Pooleery, M., Rambow, O., Roth, R.: MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic. In: Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pp. 1094–1101. Reykjavik, Iceland (2014). URL http://www.lrec-conf.org/proceedings/lrec2014/pdf/593_Paper.pdf ↗
  52. Popescu, O., Strapparava, C.: SemEval 2015, Task 7: Diachronic Text Evaluation. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pp. 870–878. Denver, Colorado (2015). URL http://www.aclweb.org/anthology/S15-2147 ↗
  53. Potthast, M., Stein, B., Barrón-Cedeño, A., Rosso, P.: An Evaluation Framework for Plagiarism Detection. In: C.R. Huang, D. Jurafsky (eds.) Proceedings of the 23rd International Conference on Computational Linguistics (COLING 2010), pp. 997–1005. COLING 2010 Organizing Committee, Beijing, China (2010) find
  54. Rashwan, M.A., Al-Badrashiny, M.A., Attia, M., Abdou, S.M., Rafea, A.: A Stochastic Arabic Diacritizer Based on a Hybrid of Factorized and Unfactorized Textual Features. Trans. Audio, Speech and Lang. Proc. 19(1), 166–175 (2011). DOI 10.1109/TASL.2010.2045240. URL http://dx.doi.org/10.1109/TASL.2010.2045240 ↗
  55. Rehurek, R., Sojka, P.: Software Framework for Topic Modelling with Large Corpora. In: Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pp. 45–50. ELRA, Valletta, Malta (2010) find
  56. Romanov, M.: Algorithmic Analysis of Medieval Arabic Biographical Collections. Speculum 92(S1), S226–S246 (2017) find
  57. Romanov, M.G.: Computational Reading of Arabic Biographical Collections with Special Reference to Preaching in the Sunni World (661–1300 CE). Ph.D. thesis, University of Michigan, Ann Arbor, MI, USA (2013) find
  58. Romeo, S., Da San Martino, G., Belinkov, Y., Barrón-Cedeño, A., Eldesouki, M., Darwish, K., Mubarak, H., Glass, J., Moschitti, A.: Language processing and learning models for community question answering in arabic. Information Processing & Management (2017). DOI https://doi.org/10.1016/j.ipm.2017.07.003 ↗
  59. Sagi, E., Kaufmann, S., Clark, B.: Semantic Density Analysis : Comparing word meaning across time and phonetic space. In: Proceedings of the EACL 2009 Workshop on GEMS: Geometrical Models of Natural Language Semantics, pp. 104–111 (2009) find
  60. Schneider, N., Mohit, B., Oflazer, K., Smith, N.A.: Coarse Lexical Semantic Annotation with Supersenses: An Arabic Case Study. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers - Volume 2, ACL ’12, pp. 253–258. Association for Computational Linguistics, Stroudsburg, PA (2012). URL http://dl.acm.org/citation.cfm?id=2390665.2390726 ↗
  61. Schönemann, P.H.: A generalized solution of the orthogonal procrustes problem. Psychometrika 31(1), 1–10 (1966). DOI 10.1007/BF02289451. URL https://doi.org/10.1007/BF02289451 ↗
  62. Shmidman, A., Koppel, M., Porat, E.: Identification of Parallel Passages Across a Large Hebrew/Aramaic Corpus. arXiv preprint arXiv:1602.08715 (2016) ↗
  63. Shoufan, A., Alameri, S.: Natural Language Processing for Dialectical Arabic: A Survey. In: Proceedings of the Second Workshop on Arabic Natural Language Processing, pp. 36–48. Beijing, China (2015). URL http://www.aclweb.org/anthology/W15-3205 ↗
  64. Smith, D.A., Cordell, R., Dillon, E.M., Stramp, N., Wilkerson, J.: Detecting and Modeling Local Text Reuse. In: Proceedings of the 14th ACM/IEEE-CS Joint Conference on Digital Libraries, JCDL ’14, pp. 183–192. London, United Kingdom (2014). URL http://dl.acm.org/citation.cfm?id=2740769.2740800 ↗
  65. Stein, B., Rosso, P., Stamatatos, E., Koppel, M., Agirre, E. (eds.): SEPLN 2009 Workshop on Uncovering Plagiarism, Authorship, and Social Software Misuse (PAN 2009), vol. 502. CEUR-WS.org, San Sebastian, Spain (2009). Http://ceur-ws.org/Vol-502 find
  66. Wijaya, D.T., Yeniterzi, R.: Understanding semantic change of words over centuries. Proceedings of the 2011 international workshop on DETecting and Exploiting Cultural diversiTy on the social web - DETECT ’11 p. 35 (2011). DOI 10.1145/2064448.2064475. URL http://dl.acm.org/citation.cfm?doid=2064448.2064475 ↗
  67. Wilkerson, J., Smith, D., Stramp, N.: Tracing the Flow of Policy Ideas in Legislatures: A Text Reuse Approach. American Journal of Political Science 59(4), 943–956 (2015). DOI 10.1111/ajps.12175. URL http://dx.doi.org/10.1111/ajps.12175 ↗
  68. Zack, E., Schippers, A. (eds.): Middle Arabic and Mixed Arabic: Diachrony and Synchrony. Brill Academic Publishers, Leiden (2012) find
  69. Zaghouani, W.: Critical Survey of the Freely Available Arabic Corpora. In: Proceedings of the Workshop on Free/Open-Source Arabic Corpora and Corpora Processing Tools (2014) find
  70. Zemánek, P., Milička, J.: Ranking Search Results for Arabic Diachronic Corpora. Google-like search engine for (non)linguists. In: Proceedings of CITALA 2014 (5th International Conference on Arabic Language Processing). Association for Computational Linguistics (2014) find
  71. Zemánek, P., Milička, J.: Quotations, Relevance and Time Depth: Medieval Arabic Literature in Grids and Networks. In: Proceedings of the 3rd Workshop on Computational Linguistics for Literature (CLFL), pp. 17–24. Gothenburg, Sweden (2014). URL http://www.aclweb.org/anthology/W14-0903 ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/belinkov-2018-studying-history-arabic-language-language/