Elamur Research⌕
☰
home / papers / 2021 / UL2C: Mapping User Locations to Countries on Arabic Twitter

UL2C: Mapping User Locations to Countries on Arabic Twitter

Paper · 2021 · Hamdy Mubarak and Sabit Hassan

Summary

Twitter lets people type any free-text location in their profile. These strings are messy: they include nicknames, informal spellings, mixed scripts, emojis and made-up places. That makes it hard to tell which country an Arabic-speaking user is from, and very few tweets carry geotags. Mubarak and Hassan built UL2C, a manually checked resource linking more than 28K unique self-declared locations to countries. The locations come from a random sample of 160K users drawn from 88M Arabic tweets collected in 2018–2020. They first labeled the strings automatically with geopy, then had a native Arabic speaker revise every one. They also compared UL2C against two other sources of training data: place names taken from GeoNames and the earlier Top10KLoc list. An SVM using character n-grams and trained on UL2C reached a macro F1 of 88.1, far above geopy and models trained on the other resources. The work shows that curated data reflecting how users actually write beats formal gazetteers. It supports dialect identification, author profiling and event detection, and a public web tool is provided.

Key points

  • Data:

    • UL2C has over 28K unique user locations.
    • About 62% of sampled users filled in a location. Roughly 60% of those entries were in Arabic and the rest mostly in English.
    • In a 500-item spot check, the authors agreed with the annotation 98% of the time.

  • Coverage:

    • About 90K users (56%) could be assigned a country. 38% had empty locations and 6% gave locations that could not be resolved.
    • Saudi Arabia accounts for over half of the users, and the Gulf region for about 70%.

  • Setup:

    • Countries outside the Arab region were merged into one UNK/OTHER class, giving 21 classes in total.
    • The models were scikit-learn LibSVM classifiers with tf-idf weighting over word [1-2] or character n-grams (up to [2-5]).

  • Results (macro F1):

    ApproachF1
    UL2C, char [2-5]88.1
    UL2C + GeoNames88.2
    Top10KLoc70.5
    geopy baseline69.2
    GeoNames (rebalanced to 8,632 entries)50.1

    The UL2C model is about 26% better in relative terms than the next-best single resource (Top10KLoc).

  • Interface: A web tool inside ASAD (asad.qcri.org) shows per-country probabilities as a map heatmap and accepts batch file uploads.

Interesting ideas

  • I1. A rough automatic tool (geopy) produces first-pass labels, and a human corrects every item, which makes annotating a large set much cheaper than labeling from scratch.
  • I2. Character n-grams handle short, noisy, mixed-script strings well, including variants a lexicon has never seen.
  • I3. A formal gazetteer (GeoNames) did poorly because people do not write places the way official lists do, so mixing in data in real usage style made the difference.
  • I4. Merging every irrelevant country into one UNK/OTHER class keeps the label space focused, and when the out-of-scope data was rebalanced against in-scope data, the classifier stopped being swamped.
  • I5. The tool outputs a full probability distribution over countries, shown as a heatmap, instead of a single label, so users can see how uncertain the system is.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Witness parsers or the LLM teacher could pre-label silver data, with humans only correcting it, which would make larger gold sets for repair rules affordable.
  • I2 (tokens): A character-n-gram fallback could guess the tag or entity type of misspelled or unseen proper names, and still explain its choice by showing the n-grams that fired.
  • I3 (world): Entity names from Wikidata/WordNet should get informal aliases and nicknames gathered from real text, so that surface mentions still link to the right entity.
  • I4 (answer): Readers could have an explicit "outside the scope of this text" answer, and evaluation sets could balance in-scope and out-of-scope questions so that refusals are measured honestly.
  • I5 (world): Ambiguous entity or location links could keep their ranked candidates with scores, so later steps and explanations can show where the uncertainty was.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksSocial Sciences · 1,772 worksTransportation · 6 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Hamdy Mubarak and Sabit Hassan. 2021. UL2C: Mapping User Locations to Countries on Arabic Twitter. Proceedings of the Sixth Arabic Natural Language Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{mubarak2021ul2c,
  title = {{UL2C: Mapping User Locations to Countries on Arabic Twitter}},
  author = {Hamdy Mubarak and Sabit Hassan},
  booktitle = {Proceedings of the Sixth Arabic Natural Language Processing Workshop},
  year = {2021},
  url = {https://aclanthology.org/2021.wanlp-1.15/}
}

References

26 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahmed Abdelali, Hamdy Mubarak, Younes Samih, Sabit Hassan, and Kareem Darwish. 2020. Arabic dialect identification in the wild. ArXiv, abs/2005.06557. ↗
  2. Muhammad Abdul-Mageed, Hassan Alhuzali, and Mohamed Elaraby. 2018. You tweet what you speak: A city-level dataset of Arabic dialects. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018), Miyazaki, Japan. European Languages Resources Association (ELRA).
  3. Puneet Agarwal, R. Vaithiyanathan, Saurabh Sharma, and G. Shroff. 2012. Catching the long-tail: Extracting local news events from twitter. In ICWSM. https://getbootstrap.com/ https://flask.palletsprojects.com/en/1.1.x/ 10 https://leafletjs.com/ 11 https://www.patrick-wied.at/static/heatmapjs/ 12 https://www.openstreetmap.org/ ↗
  4. Areej Alshutayri and Erik Atwell. 2017. Exploring twitter as a source of an arabic dialect corpus. International Journal of Computational Linguistics (IJCL), 8(2):37–44. This is an open access article under the terms of the Creative Commons Attribution License (CC-BY). find
  5. Houda Bouamor, Sabit Hassan, and Nizar Habash. 2019. The MADAR shared task on Arabic fine-grained dialect identification. In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 199–207, Florence, Italy. Association for Computational Linguistics.
  6. Kathleen M. Carley, Momin Malik, Peter M. Landwehr, Jürgen Pfeffer, and Michael Kowalchuck. 2016. Crowd sourcing disaster management: The complex nature of twitter usage in padang indonesia. Safety Science, 90:48 – 61. Building Community Resilience to Global Hazards: A Sociotechnical Approach. ↗
  7. Anis Charfi, Wajdi Zaghouani, Syed Hassan Mehdi, and Esraa Mohamed. 2019. A fine-grained annotated multi-dialectal Arabic corpus. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019), pages 198–204, Varna, Bulgaria. INCOMA Ltd.
  8. Zhiyuan Cheng, James Caverlee, and Kyumin Lee. 2010. You are where you tweet: a content-based approach to geo-locating twitter users. In CIKM ’10. find
  9. Paul Earle, Daniel Bowden, and Michelle Guy. 2012. Twitter earthquake detection: Earthquake monitoring in a social world. Annals of geophysics = Annali di geofisica, 54. ↗
  10. Mahmoud El-Haj, Paul Rayson, and Mariam Aboelezz. 2018. Arabic dialect identification in the context of bivalency and code-switching. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018), Miyazaki, Japan. European Languages Resources Association (ELRA).
  11. Sabit Hassan, Hamdy Mubarak, Ahmed Abdelali, and Kareem Darwish. 2021. Asad: Arabic social media analytics and understanding. In Proceedings of the Software Demonstrations of the 16th Conference of the European Chapter of the Association for Computational Linguistics.
  12. Sabit Hassan, Younes Samih, Hamdy Mubarak, and Ahmed Abdelali. 2020a. ALT at SemEval-2020 task 12: Arabic and English offensive language identification in social media. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 1891–1897, Barcelona (online). International Committee for Computational Linguistics.
  13. Sabit Hassan, Younes Samih, Hamdy Mubarak, Ahmed Abdelali, Ammar Rashed, and Shammur Absar Chowdhury. 2020b. ALT submission for OSACT shared task on offensive language detection. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 61–65, Marseille, France. European Language Resource Association.
  14. Brent Hecht, Lichan Hong, Bongwon Suh, and Ed H. Chi. 2011. Tweets from justin bieber’s heart: The dynamics of the location field in user profiles. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’11, page 237–246, New York, NY, USA. Association for Computing Machinery. find
  15. Binxuan Huang and Kathleen Carley. 2019. A hierarchical location prediction neural network for twitter user geolocation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4732–4742, Hong Kong, China. Association for Computational Linguistics. ↗
  16. Balachander Krishnamurthy, Phillipa Gill, and Martin Arlitt. 2008. A few chirps about twitter. pages 19– 24. find
  17. Jalal Mahmud, Jeffrey Nichols, and Clemens Drews. 2014. Home location identification of twitter users. ACM Transactions on Intelligent Systems and Technology, 5(3):1–21. ↗
  18. Yasuhide Miura, Motoki Taniguchi, Tomoki Taniguchi, and Tomoko Ohkuma. 2017. Unifying text, metadata, and user network representations with a neural network for geolocation prediction. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1260–1272, Vancouver, Canada. Association for Computational Linguistics. find
  19. Hamdy Mubarak and Ahmed Abdelali. 2016. Arabic to english person name transliteration using twitter. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 351–355. find
  20. Hamdy Mubarak, Ahmed Abdelali, Sabit Hassan, and Kareem Darwish. 2020. Spam detection on arabic twitter. In Social Informatics, pages 237–251, Cham. Springer International Publishing. find
  21. Hamdy Mubarak and Kareem Darwish. 2014. Using twitter to collect a multi-dialectal corpus of Arabic. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 1–7, Doha, Qatar. Association for Computational Linguistics.
  22. Hamdy Mubarak, Sabit Hassan, and Ahmed Abdelali. 2021. Adult content detection on arabic twitter: Analysis and experiments. In Proceedings of the Sixth Arabic Natural Language Processing Workshop. find
  23. Afshin Rahimi, Trevor Cohn, and Timothy Baldwin. 2015. Twitter user geolocation using a unified text and network prediction model. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 630–636, Beijing, China. Association for Computational Linguistics. find
  24. Afshin Rahimi, Trevor Cohn, and Timothy Baldwin. 2018. Semi-supervised user geolocation via graph convolutional networks. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2009–2019, Melbourne, Australia. Association for Computational Linguistics. find
  25. Mohammad Salameh, Houda Bouamor, and Nizar Habash. 2018. Fine-grained Arabic dialect identification. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1332–1344, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
  26. Omar F. Zaidan and Chris Callison-Burch. 2011. The Arabic online commentary dataset: an annotated dataset of informal Arabic with high dialectal content. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 37–41, Portland, Oregon, USA. Association for Computational Linguistics.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mubarak-2021-ul2c-mapping-user-locations-countries/