Elamur Research⌕
☰
home / papers / 2018 / Guidelines and Annotation Framework for Arabic Author Profiling

Guidelines and Annotation Framework for Arabic Author Profiling

Paper · 2018 · Wajdi Zaghouani and Anis Charfi

Summary

Author profiling means inferring traits such as age, gender, dialect or native language from a person's writing. It depends on texts paired with reliable author metadata, and such data was almost absent for dialectal Arabic on social media. Earlier datasets grouped the Arab world into about four broad dialect zones. Zaghouani and Charfi describe how they built and labelled the ARAP-Tweet corpus within a QNRF-funded project on profiling for cyber-security. They located public Twitter users by searching for dialect words typical of each region, then streamed tweets with a geographic filter. The tweets were cleaned of URLs, images and non-Arabic material. Annotators first decided whether each profile qualified and then labelled four attributes: gender, age band, dialect, and native versus non-native speaker. They worked from general and dialect-specific guidelines that were revised over several rounds, and quality was tracked by measuring inter-annotator agreement during the work. The outcome is a balanced resource covering 11 dialect regions across 16 countries. It gives Arabic profiling much finer regional granularity and was planned as shared-task data for PAN @ CLEF 2018.

Key points

  • Coverage: 16 of the 22 Arab League countries, grouped into 11 dialect regions. Examples of grouped regions are the Gulf (Qatari, Kuwaiti, Emirati, Saudi) and the North Levant (Lebanese, Syrian).
  • Sampling: 100 users per region, each with at least 2000 tweets. Up to 3240 recent tweets were downloaded per user, which is the Twitter API ceiling. This yielded at least 200K tweets per region.
  • The total size is given as about 2.4 million, described in places as words and in places as tweets. The data was collected with the Twitter API and TweePy over roughly six weeks of profile sampling.
  • Four labels were assigned per user: gender, dialect, native language status, and age. Age uses three bands: under 25, 25–34, and 35 or older. Age needed the most guideline revisions.
  • Annotators could consult outside sources such as personal sites, blogs, LinkedIn and Facebook. Famous and influential accounts were deliberately avoided.

Interesting ideas

  • I1. Users were collected through region-specific seed words, such as Tunisian karhba "car" or Sudanese zo:l "man". Annotators were trained to catch accounts that used a seed word but came from elsewhere, so a cheap high-recall filter is backed by a human precision check.
  • I2. Every profile first passed explicit inclusion and exclusion rules, such as being public and written mostly in dialect rather than MSA or another language, before any attribute was labelled. Separating validation from labelling keeps unusable items from polluting the labels.
  • I3. The guidelines grew iteratively with worked examples and explicit handling of borderline cases, with the most revision spent on the hardest attribute. Concentrating revision effort where agreement is weakest keeps a rulebook stable.
  • I4. Age was annotated as coarse bands rather than exact numbers, which suits evidence that is uncertain and indirect.
  • I5. Influential accounts were excluded to avoid skewing the sample toward atypical writers.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): when gathering silver sentences for a rare construction by keyword, add a verification pass by a second witness that flags keyword hits that do not really show the construction.
  • I2 (learn): run an admissibility gate on training and test items (wrong language, broken text, off-domain) before parsing or scoring them, so failures are not mistaken for parser errors.
  • I3 (tree): maintain the UD dialect-conversion rules as a living guideline with documented edge cases, and revise first the relations where witness parsers disagree most.
  • I4 (world): store vague quantities inferred from text, such as ages and durations, as bands with a provenance note instead of falsely precise values.
  • I5 (learn): build evaluation sets that exclude over-represented or famous sources, such as well-known fables, to check that the system generalises rather than recalls.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksSocial Psychology · 306 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Wajdi Zaghouani and Anis Charfi. 2018. Guidelines and Annotation Framework for Arabic Author Profiling. arXiv:1808.07678.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1808.07678 · arXiv 1808.07678

@misc{zaghouani2018guidelines,
  title = {{Guidelines and Annotation Framework for Arabic Author Profiling}},
  author = {Wajdi Zaghouani and Anis Charfi},
  year = {2018},
  doi = {10.48550/arxiv.1808.07678},
  eprint = {1808.07678},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1808.07678}
}

References

21 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abbasi A. and Chen H. (2005) Applying authorship analysis to extremistgroup web forum messages. IEEE Intelligent Systems, 20(5): 67–75. find
  2. Al-Sabbagh, Rania and Girju, Roxana. (2010). Mining the Web for the Induction of a Dialectical Arabic Lexicon. In LREC, Valetta, Malta.
  3. Boujelbane, Rahma, Ellouze Khemekhem, Mariem, and Belguith, Lamia Hadrich. (2013). Mapping Rules for Building a Tunisian Dialect Lexicon and Generating Corpora. In Proceedings of the Sixth International Joint Conference on Natural Language Processing, pages 419–428, Nagoya, Japan. find
  4. Celli, F., Pianesi, F., Stillwell, D., and Kosinski, M. (2013). Workshop on computational personality recognition (shared task). In Proceedings of the Workshop on Computational Personality Recognition, Boston, MA, USA Ramy, and Tomeh, Nadi. (2013). Morphological analysis and disambiguation for dialectal arabic. In Proceedings of NAACL-HLT, pages 426–432, Atlanta, Georgia, June.
  5. Pasha, Arfath, Al-Badrashiny, Mohamed, Kholy, Ahmed El, Eskander, Ramy, Diab, Mona, Habash, Nizar, Pooleery, Manoj, Rambow, Owen, and Roth, Ryan. (2014). Madamira: A fast, comprehensive tool for morphological analysis and disambiguation of arabic. In Proceedings of LREC, Reykjavik, Iceland.
  6. Rangel F, Rosso P, Potthast M, Stein B (2017) Overview of the 5th Author Profiling Task at PAN 2017: Gender and Language Variety Identification in Twitter. In Proceedings of PAN 2017, Dublin, Ireland Rosso P, Rangel F, Hernández Farías I, Cagnina L, Zaghouani, W, Charfi A. (2018). A survey on author profiling, deception, and irony detection for the Arabic language. Lang Linguist Compass.; https://doi.org/10.1111/lnc3.12275 ↗
  7. Sajjad, Hassan, Darwish, Kareem, and Belinkov, Yonatan. (2013). Translating dialectal Arabic to English. In Proceedings of ACL, Sofia, Bulgaria. find
  8. Salloum Wael and Nizar Habash. 2013. Dialectal Arabic to English Machine Translation: Pivoting through Modern Standard Arabic (2013). In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), Atlanta, GA.
  9. Chiang, David, Diab, Mona, Habash, Nizar, Rambow, Owen, and Shareef, Safiullah. (2006). Parsing Arabic Dialects. In Proceedings of EACL, Trento, Italy. ↗
  10. Sawaf Hassan. (2010). Arabic dialect handling in hybrid machine translation. In Proceedings of the Conference of the Association for Machine Translation in the Americas (AMTA), Denver, Colorado Diab, Mona and Habash, Nizar. (2007). Arabic Dialect Processing Tutorial. In NAACL.
  11. Zaidan, Omar and Callison-Burch, Chris. (2013). Arabic dialect identification. Computational Linguistics. ↗
  12. Elfardy, Heba and Diab, Mona. (2013). Sentence Level Dialect Identification in Arabic. In Proceedings of the Association for Computational Linguistics, pages 456– 461, Sofia, Bulgaria.
  13. Zbib, Rabih, Malchiodi, Erika, Devlin, Jacob, Stallard, David, Matsoukas, Spyros, Schwartz, Richard, Makhoul, John, Zaidan, Omar F., and CallisonBurch, Chris. (2012). Machine translation of Arabic dialects. In Proceedings of NAACL-HLT, Montréal, Canada.
  14. Estival, D., Gaustad, T., Hutchinson, B., Pham, S. B., and Radford, W. (2008): Author Profiling for English and Arabic Emails. find
  15. Natural Language Engineering, Cambridge University Press (2008). find
  16. Habash, N., Rambow, O., Diab, M., and Faraj, R. (2008). Guidelines for Annotation of Arabic Dialectness. In Proceedings of the LREC Workshop on HLT & NLP within the Arabic world. find
  17. Zaghouani, Wajdi (2014), Critical survey of the freely available Arabic corpora. In Proceedings of the 1st Workshop on Open-Source Arabic Corpora and Processing Tools, 9th International Conference on Language Resources and Evaluation, Reykjavik, Iceland. 9. ↗
  18. Language Resource References Habash, Nizar, Roth, Ryan, Rambow, Owen, Eskander, Bouamor, Houda, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdulrahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann and Kemal Oflazer. (2018). The MADAR Arabic Dialect Corpus and Lexicon. In Proceedings of the 11th International Conference on Language Habash, N. Y. (2010). Introduction to Arabic natural language processing, volume 3. Morgan & Claypool Publishers. http://edinburghnlp.inf.ed.ac.uk/workshops/OSACT3/http://pan.webis.de/clef18/pan18-web/index.html Resources and Evaluation. ↗
  19. Habash, Nizar, Salam Khalifa, Fadhl Eryani, Owen Rambow, Dana Abdulrahim, Alexander Erdmann, Reem Faraj, Wajdi Zaghouani, Houda Bouamor, Nasser Zalmout, Sara Hassan, Faisal Al shargi, Sakhar Alkhereyf, Basma Abdulkareem, Ramy Eskander, Mohammad Salameh and Hind Saddiki. (2018), Unified Guidelines and Resources for Arabic Dialect Orthography. In Proceedings of the 11th International Conference on Language Resources and Evaluation.
  20. Mubarak, H., & Darwish, K. (2014). Using Twitter to collect a multi-dialectal corpus of Arabic. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP) (pp. 1-7).
  21. Zaghouani, Wajdi and Charfi, Anis. (2018a). ArapTweet: A Large Multi-Dialect Twitter Corpus for Gender, Age and Language Variety Identification. In Proceedings of the 11th International Conference on Language Resources and Evaluation Zaghouani, Wajdi and Charfi Anis. (2018b). The ArapTweet Corpus version 0.8. Carnegie Mellon University Qatar, Doha, Qatar find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zaghouani-2018-guidelines-annotation-framework-arabic-author/