Elamur Research⌕
☰
home / papers / 2022 / Masader: Metadata Sourcing for Arabic Text and Speech Data Resources

Masader: Metadata Sourcing for Arabic Text and Speech Data Resources

Summary

Arabic NLP and speech datasets are scattered across many sites, and most come with little descriptive information. That makes it hard for researchers to learn what exists or to judge whether a dataset suits their task. The authors built Masader, a public catalogue of 200 Arabic text and speech datasets, each described by 25 metadata attributes. They found candidates with keyword searches on Google and on major hosting and indexing sites. They then kept only resources that met explicit inclusion rules and annotated the survivors, partly by hand and partly automatically, using a schema they designed. A verification pass followed before the analysis. The attributes are grouped under publication, content, accessibility, diversity and evaluation, and they cover dialect, script, volume, license, cost, domain and ethical risk. The catalogue shows recurring problems, such as missing licenses, heavy reliance on GitHub, and a strong skew toward social media text. The authors present the schema and workflow as a template that other languages, including under-documented dialects, could reuse.

Key points

  • The search, run in July–August 2021, produced about 299 candidate resources. Inclusion filtering removed 93 papers, and annotation was finished in September 2021.
  • To be included, a resource had to:
    • be text or speech,
    • have an accompanying publication,
    • date from after 2010,
    • be usable for some NLP task.
  • The schema records several practical details:
    • a size unit (tokens, sentences, documents, MB/GB/TB, hours) plus a volume count,
    • a tokenized flag,
    • a script code (Arab, Latn, Arab-Latn, Other),
    • an ethical-risk level of low, medium or high.
  • Hosting and licensing:
    • About 50.5% of the datasets live on GitHub and 23.5% on arbitrary websites.
    • Most paid resources sit at LDC.
    • More than half of the datasets declare no license.
    • About 70 distinct venues appear, with LREC and WANLP the most frequent.
  • Content and risk:
    • The domain counts are social media 61, news 24, transcribed audio 16, reviews 9, books 7 and other 83.
    • Roughly 30% of the datasets carry medium or high ethical risk.

Interesting ideas

  • I1. The schema includes a field stating whether a dataset is already tokenized, because unknown tokenization quietly changes downstream results. — For Mova (tokens): Record for every corpus and treebank we ingest whether it arrives pre-tokenized and by which tokenizer, so that tagger and lemma errors are not blamed on the wrong component.
  • I2. Ethical risk is graded by what the content may contain: personal information counts as medium, and toxic or hateful text counts as high. This gives a cheap coarse label that can be applied to every source. — For Mova (learn): Tag each raw source used for silver data or LLM-teacher prompts with such a risk tier, and filter or audit the high-tier material separately.
  • I3. The pipeline is staged as search → explicit inclusion criteria → annotate → verify → analyse, and the number of items dropped at each stage is reported. — For Mova (learn): When building silver data from agreeing parsers, log how many sentences each filter removes, so that shifts in data composition stay visible and explainable.
  • I4. The metadata uses a hierarchical taxonomy with closed vocabularies (fixed script codes and size units) instead of free-text notes. Closed vocabularies make catalogues queryable and comparable. — For Mova (tree): Keep a typed catalogue of treebanks and UD dialect conversions with fixed fields (genre, UD version, tokenization, license), so the right training mix can be chosen by query.
  • I5. Dataset abstracts were embedded with sentence-transformers and projected with t-SNE, then manually labelled clusters exposed task and format groupings. This works as a diagnostic map of a collection, not as a model. — For Mova (answer): Map our test sets (school tests, word problems, fables) this way to find which genres or question types are under-covered before adding new evaluation material.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Speech & spoken language · Semitic languages

Institutions

🎓King Fahd University of Petroleum and MineralsDhahran · Saudi Arabia23 works · 4 250 citations

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{alyafeai2022masader,
  title = {{Masader: Metadata Sourcing for Arabic Text and Speech Data Resources}},
  author = {Alyafeai, Zaid and Masoud, Maraim and Ghaleb, Mustafa and Al-shaibani, Maged S.},
  booktitle = {Proceedings of the Thirteenth Language Resources and Evaluation Conference},
  year = {2022},
  doi = {10.63317/3rot2sy5uudb},
  url = {https://doi.org/10.63317/3rot2sy5uudb}
}

References

27 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abu-Ghazaleh, T. (2000). Intellectual Property Laws of the Arab Countries. Brill, Leiden, The Netherlands. find
  2. Abu Kwaik, K., Saad, M., Chatzikyriakidis, S., and Dobnik, S. (2018). Shami: A corpus of Levantine Arabic dialects. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, May. European Language Resources Association (ELRA). find
  3. Ammar, W., Groeneveld, D., Bhagavatula, C., Beltagy, I., Crawford, M., Downey, D., Dunkelberger, J., Elgohary, A., Feldman, S., Ha, V., et al. (2018). Construction of the literature graph in semantic scholar. arXiv preprint arXiv:1805.02262. ↗ our card
  4. Antoun, W., Baly, F., and Hajj, H. (2020a). Arabert: Transformer-based model for arabic language understanding. arXiv preprint arXiv:2003.00104. ↗ our card
  5. Antoun, W., Baly, F., and Hajj, H. (2020b). Araelectra: Pre-training text discriminators for arabic language understanding. arXiv preprint arXiv:2012.15516. ↗
  6. Bender, E. M. and Friedman, B. (2018). Data statements for natural language processing: Toward mitigating system bias and enabling better science. Transactions of the Association for Computational Linguistics, 6:587–604. Darwish, K., Habash, N., Abbas, M., Al-Khalifa, H., Al-Natsheh, H. T., El-Beltagy, S., Bouamor, H., https://bigscience.huggingface.co/ ↗
  7. Bouzoubaa, K., Cavalli-Sforza, V., El-Hajj, W., Jarrar, M., and Mubarak, H. (2021). A panoramic survey of natural language processing in the arab world. Communications of the ACM, 64:72 – 81. find
  8. de Jong, F., Maegaard, B., De Smedt, K., Fišer, D., and Van Uytvanck, D. (2018). CLARIN: Towards FAIR and responsible data science using language resources. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, May. European Language Resources Association (ELRA). find
  9. Eitan, A. T., Smolyansky, E., Harpaz, I. K., and Perets, S. (2020). Connected papers: Explore connected papers in a visual graph. Accessed: 2021-10-5. find
  10. El-Haj, M. and Koulali, R. (2013). Kalimat a multipurpose arabic corpus. In Second workshop on Arabic corpus linguistics (WACL-2), pages 22–25. find
  11. Gavrilidou, M., Labropoulou, P., Desipri, E., Piperidis, S., Papageorgiou, H., Monachini, M., Frontini, F., Declerck, T., Francopoulo, G., Arranz, V., et al. (2012). The meta-share metadata schema for the description of language resources. In LREC, pages 1090–1097. find
  12. Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé III, H., and Crawford, K. (2018). Datasheets for datasets. arXiv preprint arXiv:1803.09010. ↗
  13. Guellil, I., Saâdane, H., Azouaou, F., Gueni, B., and Nouvel, D. (2021). Arabic natural language processing: An overview. Journal of King Saud University-Computer and Information Sciences, 33(5):497–507. find our card
  14. Kitchenham, B. (2004). Procedures for performing systematic reviews. Keele, UK, Keele University, 33(2004):1–26. find
  15. Labropoulou, P., Gkirtzou, K., Gavriilidou, M., Deligiannis, M., Galanis, D., Piperidis, S., Rehm, G., Berger, M., Mapelli, V., Rigault, M., et al. (2020). Making metadata fit for next generation language technology platforms: The metadata schema of the european language grid. arXiv preprint arXiv:2003.13236. ↗
  16. Mbuagbaw, L., Aves, T., Shea, B., Jull, J., Welch, V., Taljaard, M., Yoganathan, M., Greer-Smith, R., Wells, G., and Tugwell, P. (2017). Considerations and guidance in designing equity-relevant clinical trials. International journal for equity in health, 16(1):1–9. find
  17. Mohammad, S. M. (2020). NLP scholar: A dataset for examining the state of NLP research. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 868–877, Marseille, France, May. European Language Resources Association. Park, S., Moon, J., Kim, S.-D., Cho, W. I., Han, J., Park, J., Song, C., Kim, J., Song, Y., Oh, T. H., Lee, J., Oh, J., Lyu, S., kuk Jeong, Y., Lee, I., gyu Seo, S., Lee, D., Kim, H., Lee, M., Jang, S., Do, S., Kim, S., Lim, K., Lee, J., Park, K., Shin, J., Kim, S., Park, L., Oh, A. H., Ha, J.-W., and Cho, K. (2021). Klue: Korean language understanding evaluation. ArXiv, abs/2105.09680. ↗
  18. Perrier, L., Blondal, E., and MacDonald, H. (2020). The views, perspectives, and experiences of academic researchers with data sharing and reuse: A meta-synthesis. PloS one, 15(2):e0229182. find
  19. Radev, D. R., Joseph, M. T., Gibson, B., and Muthukrishnan, P. (2016). A bibliometric and network analysis of the field of computational linguistics. Journal of the Association for Information Science and Technology, 67(3):683–706. find
  20. Rowley, J. and Slack, F. (2004). Conducting a literature review. Management research news. find
  21. Seelawi, H., Tuffaha, I., Gzawi, M., Farhan, W., Talafha, B., Badawi, R., Sober, Z., Al-Dweik, O., Freihat, A. A., and Al-Natsheh, H. (2021). Alue: Arabic language understanding evaluation. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 173–184. find
  22. Sharma, A., Chhablani, G., Pandey, H., and Patil, R. (2021). Drift: A toolkit for diachronic analysis of scientific literature. arXiv preprint arXiv:2107.01198. ↗
  23. Shoufan, A. and Alameri, S. (2015). Natural language processing for dialectical Arabic: A survey. In Proceedings of the Second Workshop on Arabic Natural Language Processing, pages 36–48, Beijing, China, July. Association for Computational Linguistics. find
  24. Talafha, B., Ali, M., Za’ter, M. E., Seelawi, H., Tuffaha, I., Samir, M., Farhan, W., and Al-Natsheh, H. T. (2020). Multi-dialect arabic bert for country-level dialect identification. arXiv preprint arXiv:2007.05612. ↗
  25. Van der Maaten, L. and Hinton, G. (2008). Visualizing data using t-sne. Journal of machine learning research, 9(11). find
  26. Younes, J., Souissi, E., Achour, H., and Ferchichi, A. (2020). Language resources for maghrebi arabic dialects’ nlp: a survey. Language Resources and Evaluation, 54(4):1079–1142. find
  27. Zaghouani, W. (2017). Critical survey of the freely available arabic corpora. CoRR, abs/1702.07835. ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alyafeai-2022-masader-metadata-sourcing-arabic-text/