Elamur Research⌕
☰
home / papers / 2019 / Classifying Information Sources in Arabic Twitter to Support Online Monitoring of Infectious Diseases

Classifying Information Sources in Arabic Twitter to Support Online Monitoring of Infectious Diseases

Paper · 2019 · Lama Alsudias and Paul Rayson

Summary

Arabic tweets about health spread both useful guidance and a lot of misinformation. Earlier monitoring work has mostly examined English tweets and looked only at what tweets say. Alsudias and Rayson instead ask who posted the information, since the type of source signals how much a reader can trust it.

They gathered 10,000 Arabic tweets between December 2018 and February 2019, using keywords taken from a translated English Disease Ontology. After manual filtering and cleaning, 1,266 tweets remained. Each was labelled by its author type: academic, media, government, health professional or public. Two native speakers labelled the data independently, reaching an agreement of 0.84.

Four classical classifiers were then trained with 10-fold cross-validation. Using tweet text alone, they reached about 77% accuracy. Adding the user's profile bio pushed accuracy to roughly 99.9%, though the authors attribute this partly to the small number of accounts in the data.

The paper's contribution is a freely released dataset plus evidence that source type is learnable from text. This makes source type a practical first step toward filtering health information by reliability.

Key points

  • Collection: Tweets were found using keywords translated from the English Disease Ontology, because no suitable Arabic ontology was available. Collection ran in two passes:
    • a keyword search, which also surfaced relevant accounts;
    • a crawl of those accounts' history, filtered by the same keywords.
  • Dataset: 1,266 tweets and 18,985 words after filtering. Class sizes are uneven: academic 239, media 56, government 258, health professional 277, public 436.
  • Models: Logistic Regression, Random Forest, Multinomial Naïve Bayes and Linear SVC. Features were chosen with univariate statistical tests or with model-based selection.
  • Results:
    • Text-only accuracy peaked at 77.2% (LR).
    • Over-sampling plus best-feature selection raised LR to 79.1%.
    • Under-sampling hurt accuracy.
    • Adding the bio gave 99.6–99.9% for every model.
    • The most frequent confusion was between academic and government tweets.
  • Release: The data, including tweet IDs and labels, is available at doi 10.17635/lancaster/researchdata/303.

Interesting ideas

  • I1. When the target language has no domain ontology, translating an English one gives a usable seed vocabulary with synonyms already grouped.
  • I2. Keyword search can discover productive sources, and each source's full history can then be harvested and filtered by the same keywords, which works around API limits on old posts.
  • I3. Before training anything, check with two independent annotators that the label scheme is reproducible by humans.
  • I4. The near-perfect bio score is explained by the small number of accounts, which is a reminder that metadata can let a model memorise sources rather than generalise.
  • I5. Treating the speaker's type as a reliability signal alongside the content of a claim is a simple but meaningful design choice.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): Missing domain lexicons in compiled knowledge could be bootstrapped from an existing ontology and then hand-checked, instead of being built from scratch.
  • I2 (learn): Silver-data collection could first find texts rich in a target construction, then mine their whole origin (author or collection) for more examples of the same kind.
  • I3 (learn): New frame or speech-act label sets should pass an inter-annotator test before LLM teachers or witness parsers produce silver data with them.
  • I4 (learn): Evaluations should split by document or source and include a negative control, so that apparent gains are not just identity leakage.
  • I5 (frames): Speech-act frames could carry a source-type slot (official, expert, layperson), so that the world model weights or flags claims by who asserted them.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksSocial Sciences · 1,772 worksSociology and Political Science · 577 worksMedicine · 241 worksEpidemiology · 7 works

🧭 Topics 1Research subjects and languages the work is about.Semitic languages · 2,908 works

Cite

Cite as Lama Alsudias and Paul Rayson. 2019. Classifying Information Sources in Arabic Twitter to Support Online Monitoring of Infectious Diseases. Proceedings of the 3rd Workshop on Arabic Corpus Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{alsudias2019classifying,
  title = {{Classifying Information Sources in Arabic Twitter to Support Online Monitoring of Infectious Diseases}},
  author = {Lama Alsudias and Paul Rayson},
  booktitle = {Proceedings of the 3rd Workshop on Arabic Corpus Linguistics},
  year = {2019},
  url = {https://aclanthology.org/W19-5604/}
}

References

22 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Wasim Ahmed, Peter Bath, Laura Sbaffi, and Gianluca Demartini. 2018. Moral panic through the lens of twitter: An analysis of infectious disease outbreaks. In Proceedings of the 9th International Conference on Social Media and Society, pages 217–221. ACM. find
  2. Abdulaziz Alayba, Vasile Palade, Matthew England, and Rahat Iqbal. 2017. Arabic language sentiment analysis on health services. In 2017 1st International Workshop on Arabic Script Analysis and Recognition (ASAR), pages 114–118.
  3. Khalid Alnemer, Waleed Alhuzaim, Ahmed Alnemer, Bader Alharbi, Abdulrahman Bawazir, Omar Barayyan, and Faisal Balaraj. 2015. Are health-related tweets evidence based? review and analysis of health-related tweets on twitter. Journal of medical Internet research, 17(10). ↗
  4. Hana Alsobayel. 2016. Use of social media for professional development by health care professionals: A cross-sectional web-based survey. JMIR Med Educ, 2(2):e15. ↗
  5. Eiji Aramaki, Sachiko Maskawa, and Mizuki Morita. 2011. Twitter catches the flu: detecting influenza epidemics using twitter. In Proceedings of the conference on empirical methods in natural language processing, pages 1568–1576. Association for Computational Linguistics. find
  6. Ron Artstein and Massimo Poesio. 2008. Inter-coder agreement for computational linguistics. Computational Linguistics, 34(4):555–596. ↗
  7. Cesar Arze, Gang Feng, Mark Mazaitis, Suvarna Nadendla, Victor Felix, Yu-Wei Wayne Chang, Lynn Marie Schriml, and Warren Alden Kibbe. 2011. Disease Ontology: a backbone for disease semantic integration. Nucleic Acids Research, 40(D1):D940–D946. find
  8. Jessica Breland, Lisa Quintiliani, Kristin Schneider, Christine May, and Sherry Pagoto. 2017. Social media as a tool to increase the impact of public health research. American Journal of Public Health, 107(12):1890–1891. PMID: 29116846. ↗
  9. Evgeny Burnaev, Pavel Erofeev, and Artem Papanov. 2015. Influence of resampling on accuracy of imbalanced classification. In Eighth International Conference on Machine Vision (ICMV 2015), volume 9875, page 987521. International Society for Optics and Photonics. find
  10. Lauren Charles-Smith, Tera Reynolds, Mark Cameron, Mike Conway, Eric HY Lau, Jennifer Olsen, Julie Pavlin, Mika Shigematsu, Laura Streichert, Katie Suda, et al. 2015. Using social media for actionable disease surveillance and outbreak management: a systematic literature review. PloS one, 10(10):e0139701. ↗
  11. Son Doan, Elly W Yang, Sameer Tilak, and Manabu Torii. 2018. Using natural language processing to extract health-related causality from twitter messages. In 2018 IEEE International Conference on Healthcare Informatics Workshop (ICHI-W), pages 84–85. IEEE. find
  12. Yang Hong and Richard Sinnott. 2018. A social media platform for infectious disease analytics. In International Conference on Computational Science and Its Applications, pages 526–540. Springer. find
  13. Xiang Ji, Soon Ae Chun, and James Geller. 2013. Monitoring public health concerns using twitter sentiment classifications. In Healthcare Informatics (ICHI), 2013 IEEE International Conference on, pages 335–344. IEEE. find
  14. Xiang Ji, Soon Ae Chun, and James Geller. 2016. Knowledge-based tweet classification for disease sentiment monitoring. In Sentiment Analysis and Ontology Engineering, pages 425–454. Springer. ↗
  15. Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Association for Computational Linguistics (ACL) System Demonstrations, pages 55–60.
  16. Michael Paul and Mark Dredze. 2011. You are what you tweet: Analyzing twitter for public health. Icwsm, 20:265–272. find
  17. Michael Paul, Abeed Sarker, John Brownstein, Azadeh Nikfarjam, Matthew Scotch, Karen Smith, and Graciela Gonzalez. 2016. Social media mining for public health monitoring and surveillance. In Biocomputing 2016: Proceedings of the Pacific Symposium, pages 468–479. World Scientific. Fabian Pedregosa, Gaël. find
  18. Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Édouard Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830. find
  19. Lynn Marie Schriml, Cesar Arze, Suvarna Nadendla, Yu-Wei Wayne Chang, Mark Mazaitis, Victor Felix, Gang Feng, and Warren Alden Kibbe. 2011. Disease ontology: a backbone for disease semantic integration. Nucleic acids research, 40(D1):D940–D946. ↗
  20. Lauren Sinnenberg, Alison Buttenheim, Kevin Padrez, Christina Mancheno, Lyle Ungar, and Raina Merchant. 2017. Twitter as a tool for health research: A systematic review. American Journal of Public Health, 107(1):e1–e8. PMID: 27854532. ↗
  21. Si Sivasankari, Mu Kavitha, and Gi Saranya. 2017. Medical analysis and visualisation of diseases using tweet data. Research Journal of Pharmacy and Technology, 10(12):4306–4312. ↗
  22. Antonio Yepes, Andrew MacKinlay, and Bo Han. 2015. Investigating public health surveillance using twitter. Proceedings of BioNLP 15, pages 164–170. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alsudias-2019-classifying-information-sources-arabic-twitter/