Elamur Research⌕
☰
home / papers / 2016 / Sentiment Lexicons for Arabic Social Media

Sentiment Lexicons for Arabic Social Media

Paper · 2016 · Saif Mohammad, Mohammad Salameh and Svetlana Kiritchenko

Summary

Arabic sentiment analysis lacks large polarity lexicons. Earlier Arabic resources hold from a few hundred to a few thousand words, and most give no strength score. The authors built much larger lexicons in two ways. First, they mined Arabic tweets with distant supervision: emoticons or seed hashtags served as noisy positive and negative labels, and each word got a PMI-difference score. Second, they translated six well-known English lexicons word by word with Google Translate. To test the lexicons, they rebuilt the NRC-Canada sentence classifier for Arabic and ran it on 1,200 crowd-annotated sentences from the BBN Arabic Dialectal Text, which mixes Levantine and MSA. Lexicon features improved on a strong surface-feature baseline. The dialectal hashtag lexicon helped most, and combining it with the translated NRC Emotion Lexicon gave the best accuracy. A manual check found that most translated entries keep their polarity. The error analysis shows where translation fails. The resources are released freely and give dialect-heavy social media text far better coverage.

Key points

  • Three new tweet-derived lexicons, each scored as PMI(w,pos) − PMI(w,neg) over words seen at least five times:
    • Arabic Emoticon Lexicon: 43,304 entries, from close to one million tweets containing ":)" or ":(".
    • Arabic Hashtag Lexicon: 21,964 entries, from 77 seeds translated out of the NRC Hashtag Sentiment Lexicon.
    • Arabic Hashtag Lexicon (dialectal): 20,128 entries, seeded with the 483 dialectal words of Refaee and Rieser (2014).
  • Six English lexicons were translated into Arabic: AFINN, Bing Liu, MPQA, NRC Emotion, NRC Emoticon (Sentiment140) and NRC Hashtag. Some words could not be translated.
  • The classifier was a linear SVM (LIBSVM) with word/character n-grams and other surface features. Preprocessing used the CMU Twitter tokenizer, MADA lemmas, and Alif/Ya normalization.
  • Ten-fold cross-validation accuracy was 62.0 for the baseline and 62.2–63.0 with existing manual lexicons. The dialectal hashtag lexicon reached 65.3, the best translated lexicon (NRC Emotion) 63.5, and the two combined 66.6.
  • A native-speaker review rated about 88% of translated entries as polarity-correct. Among the faulty ones, roughly 10% were gross mistranslations, 40% were renderings into a related word, and 50% reflected different usage in Arabic.

Interesting ideas

  • I1. Reusing cheap signals already in the text (emoticons, hashtags) as weak labels, then scoring words by the PMI gap between the two classes with a minimum-frequency cutoff, yields large graded lexicons without any annotation.
  • I2. A small seed list matched to the target register (dialectal words) beat bigger but generic or translated resources, so domain fit of seeds mattered more than lexicon size.
  • I3. Each resource was evaluated by adding it alone to one fixed baseline, which shows clearly how much each lexicon contributes and exposes resources that add nothing.
  • I4. Translated entries were sorted into three failure types: gross mistranslation, shift to a related word, and culture-driven usage differences. This taxonomy explains why meaning drifts across languages.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): A similar PMI-gap score over weakly labelled English text could seed graded polarity values in the compiled knowledge layer, with each value traceable to counts.
  • I2 (learn): Silver-data bootstrapping should begin from seeds sampled from the genre being parsed, such as fables or school texts, not from generic word lists.
  • I3 (learn): Adding one knowledge source at a time against a frozen baseline, paired with negative controls, would show which compiled resources actually pay off.
  • I4 (back): Translation through the meta-world could tag each lexical choice with these categories and flag cases where the evaluative charge of a word may not survive.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏛️National Research Council CanadaOttawa · Canada59 works · 21,677 citations🎓University of AlbertaEdmonton · Canada44 works · 3,236 citations🏛️National Academies of Sciences, Engineering, and MedicineWashington · United States12 works · 2,981 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 works

Cite

Cite as Saif Mohammad, Mohammad Salameh and Svetlana Kiritchenko. 2016. Sentiment Lexicons for Arabic Social Media. Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{mohammad2016sentiment,
  title = {{Sentiment Lexicons for Arabic Social Media}},
  author = {Saif Mohammad, Mohammad Salameh and Svetlana Kiritchenko},
  booktitle = {Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)},
  year = {2016},
  doi = {10.63317/28kytu8w5m87},
  url = {https://doi.org/10.63317/28kytu8w5m87}
}

References

24 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdul-Mageed, M., Diab, M. T., and Korayem, M. (2011). Subjectivity and sentiment analysis of Modern Standard Arabic. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, pages 587–591, Portland, OR.
  2. Badaro, G., Baly, R., Hajj, H., Habash, N., and El-Hajj, W. (2014). A large scale Arabic sentiment lexicon for Arabic opinion mining. In Proceedings of the EMNLP Workshop on Arabic Natural Language Processing (ANLP), pages 165–173, Doha, Qatar.
  3. Chang, C.-C. and Lin, C.-J. (2011). LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(3):27:1–27:27. find
  4. Farra, N., Challita, E., Assi, R. A., and Hajj, H. (2010). Sentence-level and document-level sentiment mining for Arabic texts. In Proceedings of the IEEE International Conference on Data Mining Workshops, pages 1114– 1119. IEEE. find
  5. Go, A., Bhayani, R., and Huang, L. (2009). Twitter sentiment classification using distant supervision. Technical report, Stanford University. find
  6. Habash, N., Rambow, O., and Roth, R. (2009). MADA+TOKAN: A toolkit for Arabic tokenization, diacritization, morphological disambiguation, POS tagging, stemming and lemmatization. In Proceedings of the 2nd International Conference on Arabic Language Resources and Tools, pages 102–109, Cairo, Egypt. find
  7. Hu, M. and Liu, B. (2004). Mining and summarizing customer reviews. In Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’04, pages 168–177, New York, NY, USA. find
  8. Kiritchenko, S., Zhu, X., Cherry, C., and Mohammad, S. (2014a). NRC-Canada-2014: Detecting aspects and sentiment in customer reviews. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 437–442, Dublin, Ireland, August.
  9. Kiritchenko, S., Zhu, X., and Mohammad, S. M. (2014b). Sentiment analysis of short informal texts. Journal of Artificial Intelligence Research, 50:723–762. ↗
  10. Kiritchenko, S., Mohammad, S. M., and Salameh, M. (2016). SemEval-2016 Task 7: Determining sentiment intensity of English and Arabic phrases. In Proceedings of the International Workshop on Semantic Evaluation, SemEval ’16, San Diego, California.
  11. Mohammad, S. M. and Turney, P. D. (2010). Emotions evoked by common words and phrases: Using Mechanical Turk to create an emotion lexicon. In Proceedings of the NAACL-HLT Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, pages 26–34, LA, California.
  12. Mohammad, S. M. and Turney, P. D. (2013). Crowdsourcing a word-emotion association lexicon. Computational Intelligence, 29(3):436–465. ↗
  13. Mohammad, S. M., Kiritchenko, S., and Zhu, X. (2013). NRC-Canada: Building the state-of-the-art in sentiment analysis of tweets. In Proceedings of the 7th International Workshop on Semantic Evaluation Exercises, SemEval ’13, Atlanta, Georgia, USA, June. ↗
  14. Mohammad, S. M., Kiritchenko, S., Sobhani, P., Zhu, X., and Cherry, C. (2016a). SemEval-2016 Task 6: Detecting stance in tweets. In Proceedings of the International Workshop on Semantic Evaluation, SemEval ’16, San Diego, California, June.
  15. Mohammad, S. M., Salameh, M., and Kiritchenko, S. (2016b). How translation alters sentiment. Journal of Artificial Intelligence Research, 55:95–130. ↗
  16. Nielsen, F. Å. (2011). A new ANEW: Evaluation of a word list for sentiment analysis in microblogs. In Proceedings of the ESWC2011 Workshop on ’Making Sense of Microposts’: Big things come in small packages, pages 93–98. ↗
  17. Pontiki, M., Galanis, D., Pavlopoulos, J., Papageorgiou, H., Androutsopoulos, I., and Manandhar, S. (2014). SemEval-2014 Task 4: Aspect based sentiment analysis. In Proceedings of the International Workshop on Semantic Evaluation, SemEval ’14, pages 27–35, Dublin, Ireland, August.
  18. Refaee, E. and Rieser, V. (2014). An Arabic Twitter corpus for subjectivity and sentiment analysis. In Proceedings of the 9th International Conference on Language Resources and Evaluation, Reykjavik, Iceland.
  19. Rosenthal, S., Nakov, P., Kiritchenko, S., Mohammad, S. M., Ritter, A., and Stoyanov, V. (2015). SemEval-2015 Task 10: Sentiment analysis in Twitter. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 451–463, Denver, Colorado, June.
  20. Salameh, M., Mohammad, S. M., and Kiritchenko, S. (2015). Sentiment after translation: A case-study on Arabic social media posts. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 767–777, Denver, Colorado.
  21. Wilson, T., Wiebe, J., and Hoffmann, P. (2005). Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, HLT ’05, pages 347–354, Stroudsburg, PA, USA.
  22. Wilson, T., Kozareva, Z., Nakov, P., Rosenthal, S., Stoyanov, V., and Ritter, A. (2013). SemEval-2013 Task 2: Sentiment analysis in Twitter. In Proceedings of the International Workshop on Semantic Evaluation, SemEval ’13, Atlanta, Georgia, USA, June.
  23. Zbib, R., Malchiodi, E., Devlin, J., Stallard, D., Matsoukas, S., Schwartz, R., Makhoul, J., Zaidan, O. F., and Callison-Burch, C. (2012). Machine translation of Arabic dialects. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 49–59.
  24. Zhu, X., Kiritchenko, S., and Mohammad, S. (2014). NRC-Canada-2014: Recent improvements in the sentiment analysis of tweets. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 443–447, Dublin, Ireland, August. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mohammad-2016-sentiment-lexicons-arabic-social-media/