Elamur Research⌕
☰
home / papers / 2019 / ArSentD-LEV: A Multi-Topic Corpus for Target-based Sentiment Analysis in Arabic Levantine Tweets

ArSentD-LEV: A Multi-Topic Corpus for Target-based Sentiment Analysis in Arabic Levantine Tweets

Summary

Most Arabic sentiment datasets assign only a single polarity label to a whole text, often on a coarse three-way scale. They also overlook dialect and topic, both of which shift how opinions are expressed. Baly et al. address this for the Levantine dialect by building ArSenTD-LEV, a set of 4,000 tweets from Jordan, Lebanon, Palestine and Syria. Before collecting data, the authors manually examined a pilot sample of 200 Levantine tweets. From it they learned which topics dominate, how language is used, and how sentiment and its targets appear. They then used it to derive topic keywords for retrieval. Crowd workers labelled each tweet with four things: overall sentiment on a five-point polarity/intensity scale, the target of that sentiment, whether it was stated explicitly or implicitly, and the topic. Baseline experiments indicate that topic-aware models beat topic-agnostic ones and that accuracy drops when training and test topics differ. The resource matters because it combines, in one dialectal corpus, annotation layers that earlier Arabic datasets offered only separately or not at all.

Key points

  • ArSenTD-LEV contains 4,000 Levantine tweets. Each has a 5-point sentiment label (very negative to very positive), a sentiment target, an explicit/implicit flag, and a topic label. The corpus is publicly released through the OMA project site.
  • Topic shares in the 200-tweet pilot sample were: personal 36%, politics 23%, religion 11%, sports 6%, other 24%.
  • Annotation was crowdsourced, and the guidelines came from the manual pilot analysis.
  • Some phrases are culture-dependent. An expression glorifying God reads as positive in the Levant but as a neutral religious formula in the Gulf.
  • Experiments show topic-based classifiers outperform topic-blind ones, and there is a clear cross-domain performance gap.

Interesting ideas

  • I1. Run a small manual pilot analysis before writing annotation guidelines, so that label categories reflect what the data actually contains. — For Mova (learn): Before building a new silver or evaluation set, hand-inspect a few hundred sentences to decide which phenomena the labels must cover.
  • I2. Attach sentiment to an explicit target instead of to the whole text, so one sentence can hold several opinions about different things. — For Mova (frames): Represent attitudes as frame relations (experiencer → stance → target) in prag, so it can explain who likes or dislikes what.
  • I3. Mark whether an opinion is stated outright or only implied, treating implicitness as its own annotation dimension. — For Mova (frames): An explicit/implicit flag on evaluative frames would link naturally to the implicature and irony modules and show which inference step produced the attitude.
  • I4. Note that a fixed phrase can carry different evaluative force depending on region or culture. — For Mova (world): Tag compiled lexical knowledge (e.g., polarity of idioms) with a register or region attribute, rather than storing one universal value.
  • I5. Test models by training on one topic and evaluating on another to expose domain dependence. — For Mova (learn): Add cross-genre splits (fables vs. school texts vs. news) as standard evaluation controls to detect parsers or readers that overfit one domain.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

26 papers of our library cite it (of the 140884 whose bibliographies we read); 10 of them have a page here, 16 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1906.01830 · arXiv 1906.01830

@misc{baly2019arsentd,
  title = {{ArSentD-LEV: A Multi-Topic Corpus for Target-based Sentiment Analysis in Arabic Levantine Tweets}},
  author = {Baly, Ramy and Khaddaj, Alaa and Hajj, Hazem and El-Hajj, Wassim and Science, Khaled Bashir Shaban   MIT Computer and Laboratory, Artificial Intelligence and Cambridge and MA and USA and Beirut, American University of and Electrical and Department, Computer Engineering and Beirut and Lebanon and Beirut, American University of and Department, Computer Science and Beirut and Lebanon and University, Qatar and Science, Computer and Department, Engineering and Doha and Qatar)},
  year = {2019},
  doi = {10.48550/arxiv.1906.01830},
  eprint = {1906.01830},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1906.01830}
}

References

49 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdul-Mageed, M. and Diab, M. T. (2012). Awatif: A multi-genre corpus for modern standard arabic subjectivity and sentiment analysis. In LREC, pages 3907–3914. find
  2. Abdul-Mageed, M., Diab, M. T., and Korayem, M. (2011). Subjectivity and sentiment analysis of modern standard arabic. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers-Volume 2, pages 587–591. Association for Computational Linguistics. find
  3. Abdul-Mageed, M., Diab, M., and Kübler, S. (2014). Samar: Subjectivity and sentiment analysis for arabic social media. Computer Speech & Language, 28(1):20–37. find
  4. Al-Kabi, M., Al-Ayyoub, M., Alsmadi, I., and Wahsheh, H. (2016). A prototype for a standard arabic sentiment analysis corpus. Int. Arab J. Inf. Technol., 13(1A):163– 170. find
  5. Al Sallab, A. A., Baly, R., Badaro, G., Hajj, H., El Hajj, W., and Shaban, K. B. (2015). Deep learning models for sentiment analysis in arabic. In ANLP Workshop, volume 9. find
  6. Al-Sallab, A., Baly, R., Hajj, H., Shaban, K. B., El-Hajj, W., and Badaro, G. (2017). Aroma: A recursive deep learning model for opinion mining in arabic as a low resource language. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 16(4):25. find
  7. Al-Smadi, M., Qawasmeh, O., Talafha, B., and Quwaider, M. (2015). Human annotated arabic dataset of book reviews for aspect based sentiment analysis. In Future Internet of Things and Cloud (FiCloud), 2015 3rd International Conference on, pages 726–730. IEEE. find
  8. Al-Twairesh, N., Al-Khalifa, H., Al-Salman, A., and Al-Ohali, Y. (2017). Arasenti-tweet: A corpus for arabic sentiment analysis of saudi tweets. Procedia Computer Science, 117:63–72. find
  9. Aly, M. A. and Atiya, A. F. (2013). Labr: A large scale arabic book reviews dataset. In ACL (2), pages 494–498. find
  10. Assiri, A., Emam, A., and Al-Dossari, H. (2016). Saudi twitter corpus for sentiment analysis. World Academy of Science, Engineering and Technology, International Journal of Computer, Electrical, Automation, Control and Information Engineering, 10(2):272–275. find
  11. Badaro, G., Baly, R., Hajj, H., Habash, N., and El-Hajj, W. (2014). A large scale arabic sentiment lexicon for arabic opinion mining. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 165–173. find
  12. Badaro, G., Baly, R., Akel, R., Fayad, L., Khairallah, J., Hajj, H., El-Hajj, W., and Shaban, K. B. (2015). A light lexicon-based mobile application for sentiment mining of arabic tweets. In ANLP Workshop 2015, page 18. find
  13. Baly, R., Hobeica, R., Hajj, H., El-Hajj, W., Shaban, K. B., and Al-Sallab, A. (2016). A meta-framework for modeling the human reading process in sentiment analysis. ACM Transactions on Information Systems (TOIS), 35(1):7. find
  14. Baly, R., Badaro, G., El-Khoury, G., Moukalled, R., Aoun, R., Hajj, H., El-Hajj, W., Habash, N., and Shaban, K. B. (2017a). A characterization study of arabic twitter data with a benchmarking for state-of-the-art opinion mining models. WANLP 2017 (co-located with EACL 2017), page 110. find
  15. Baly, R., Badaro, G., Hamdi, A., Moukalled, R., Aoun, R., El-Khoury, G., Al Sallab, A., Hajj, H., Habash, N., Shaban, K., et al. (2017b). Omam at semeval-2017 task 4: Evaluation of english state-of-the-art sentiment analysis models for arabic and a new topic-based model. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 603–610. find
  16. Baly, R., El-Khoury, G., Moukalled, R., Aoun, R., Hajj, H., Shaban, K. B., and El-Hajj, W. (2017c). Comparative evaluation of sentiment analysis methods across arabic dialects. Procedia Computer Science, 117:266–273. find
  17. Baly, R., Hajj, H., Habash, N., Shaban, K. B., and El-Hajj, W. (2017d). A sentiment treebank and morphologically enriched recursive deep models for effective sentiment analysis in arabic. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 16(4):23. find
  18. Biyani, P., Caragea, C., and Bhamidipati, N. (2015). Entity-specific sentiment classification of yahoo news comments. arXiv preprint arXiv:1506.03775. ↗
  19. Brody, S. and Elhadad, N. (2010). An unsupervised aspect-sentiment model for online reviews. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 804–812. Association for Computational Linguistics. find our card
  20. Chen, X., Sun, Y., Athiwaratkun, B., Cardie, C., and Weinberger, K. (2016). Adversarial deep averaging networks for cross-lingual sentiment classification. arXiv preprint arXiv:1606.01614. ↗ our card
  21. ElSahar, H. and El-Beltagy, S. R. (2015). Building large arabic multi-domain resources for sentiment analysis. In CICLing (2), pages 23–34. find
  22. Farra, N. and McKeown, K. (2017). Smarties: Sentiment models for arabic target entities. arXiv preprint arXiv:1701.03434. ↗
  23. Farra, N., Challita, E., Assi, R. A., and Hajj, H. (2010). Sentence-level and document-level sentiment mining for arabic texts. In Data Mining Workshops (ICDMW), 2010 IEEE International Conference on, pages 1114–1119. IEEE. find
  24. Jiang, L., Yu, M., Zhou, M., Liu, X., and Zhao, T. (2011). Target-dependent twitter sentiment classification. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 151–160. Association for Computational Linguistics. find
  25. Joty, S., Nakov, P., Màrquez, L., and Jaradat, I. (2017). Cross-language learning with adversarial neural networks. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 226–237. find
  26. Kirange, D. and Deshmukh, R. R. (2015). Emotion classification of restaurant and laptop review dataset: Semeval 2014 task 4. International Journal of Computer Applications, 113(6). find
  27. Li, Z. (2017). End-to-end adversarial memory network for cross-domain sentiment classification. find
  28. Liu, B. (2012). Sentiment analysis and opinion mining. Synthesis lectures on human language technologies, 5(1):1–167. find
  29. Maamouri, M., Bies, A., Buckwalter, T., and Mekki, W. (2004). The penn arabic treebank: Building a large-scale annotated arabic corpus. In NEMLAR conference on Arabic language resources and tools, volume 27, pages 466–467. find
  30. Medhaffar, S., Bougares, F., Esteve, Y., and Hadrich-Belguith, L. (2017). Sentiment analysis of tunisian dialects: Linguistic ressources and experiments. In Proceedings of the Third Arabic Natural Language Processing Workshop, pages 55–61. find
  31. Mohammad, S. M., Salameh, M., and Kiritchenko, S. (2016). How translation alters sentiment. J. Artif. Intell. Res.(JAIR), 55:95–130. find
  32. Mohit, B., Rozovskaya, A., Habash, N., Zaghouani, W., and Obeid, O. (2014). The first qalb shared task on automatic text correction for arabic. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 39–47. find
  33. Nabil, M., Aly, M. A., and Atiya, A. F. (2015). Astd: Arabic sentiment tweets dataset. In EMNLP, pages 2515– 2519. find our card
  34. Pan, S. J., Ni, X., Sun, J.-T., Yang, Q., and Chen, Z. (2010). Cross-domain sentiment classification via spectral feature alignment. In Proceedings of the 19th international conference on World wide web, pages 751–760. ACM. find
  35. Pang, B., Lee, L., and Vaithyanathan, S. (2002). Thumbs up?: sentiment classification using machine learning techniques. In Proceedings of the ACL-02 conference on Empirical methods in natural language processing-Volume 10, pages 79–86. Association for Computational Linguistics. find our card
  36. Paolillo, J. C. and Das, A. (2006). Evaluating language statistics: The ethnologue and beyond. Contract report for UNESCO Institute for Statistics. find
  37. Ravi, K. and Ravi, V. (2015). A survey on opinion mining and sentiment analysis: tasks, approaches and applications. Knowledge-Based Systems, 89:14–46. find
  38. Refaee, E. and Rieser, V. (2014a). An arabic twitter corpus for subjectivity and sentiment analysis. In LREC, pages 2268–2273. find
  39. Refaee, E. and Rieser, V. (2014b). Subjectivity and sentiment analysis of arabic twitter feeds with limited resources. In Workshop on Free/Open-Source Arabic Corpora and Corpora Processing Tools Workshop Programme, page 16. find
  40. Rosenthal, S., Farra, N., and Nakov, P. (2017). Semeval-2017 task 4: Sentiment analysis in twitter. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 502–518. find our card
  41. Rushdi-Saleh, M., Martı́n-Valdivia, M. T., Ureña-López, L. A., and Perea-Ortega, J. M. (2011). Oca: Opinion corpus for arabic. Journal of the Association for Information Science and Technology, 62(10):2045–2054. find
  42. Salameh, M., Mohammad, S., and Kiritchenko, S. (2015). Sentiment after translation: A case-study on arabic social media posts. In HLT-NAACL, pages 767–777. find
  43. Salem, F. (2017). Social media and the internet of things towards data-driven policymaking in the arab world: Potential, limits and concerns. find
  44. Sareah, F. (2015). Interesting statistics for the top 10 social media sites. Small Business Trends. find
  45. Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., and Potts, C. (2013). Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1631–1642. find
  46. Somasundaran, S. and Wiebe, J. (2009). Recognizing stances in online debates. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1-Volume 1, pages 226–234. Association for Computational Linguistics. find
  47. Tang, D., Qin, B., and Liu, T. (2015). Document modeling with gated recurrent neural network for sentiment classification. In EMNLP, pages 1422–1432. find our card
  48. Wang, Y., Huang, M., Zhu, X., and Zhao, L. (2016). Attention-based lstm for aspect-level sentiment classification. In EMNLP, pages 606–615. find our card
  49. Zaidan, O. F. and Callison-Burch, C. (2014). Arabic dialect identification. Computational Linguistics, 40(1):171–202. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/baly-2019-arsentd-lev-multi-topic-corpus/