Elamur Research⌕
☰
home / papers / 2017 / SMARTies: Sentiment Models for Arabic Target entities

SMARTies: Sentiment Models for Arabic Target entities

Paper · 2017 · Noura Farra and Kathy McKeown

Summary

The paper addresses open-domain targeted sentiment in Arabic: given a long, informal reader comment, find which entities the writer holds opinions about and whether each opinion is positive or negative. Targets may be any noun phrase, including events and concepts, but must appear explicitly in the text. The authors use the Arabic Opinion Target dataset, which consists of Aljazeera comments drawn from the QALB corpus. They build two separate linear-chain CRF taggers. The first marks target tokens. The second assigns polarity, using the target labels as an input feature. Both models draw on Arabic morphology (clitic segmentation schemes, lemmas, detailed POS), sentiment lexicons, syntax, and word-vector clusters trained on a large Arabic corpus. The resulting system, SMARTies, clearly beats several lexical baselines and is roughly in line with comparable English work. The study shows that the choice of segmentation affects target finding and polarity differently. It also shows that distributional clusters alone come close to a resource-heavy linguistic model, which makes them attractive for low-resource languages.

Key points

  • The data has 1177 comments of 1–3 sentences, averaging 51 words, covering politics, culture and sports. It contains 4886 targets: 38.2% positive, 50.5% negative and 11.3% ambiguous. The split is 80/10/10 for train, dev and blind test.
  • The target model labels tokens T/O. The sentiment model labels them P/N/∅ under a constraint: target tokens must be P or N, and all other tokens are ∅. Ambiguous-polarity targets are used to train the target model only.
  • Splitting off the definite article Al+ raises target recall, while the sentiment model does better with less segmentation. The best setups combine D3 and D3+ATB schemes with clusters (k=8000), and blind-test results mirror the dev results.
  • The lexicons considered include SIFAAT (3982 manually built adjectives), ArSenL, and MPQA matched through English glosses.
  • In a manual check of 50 dev posts, the primary target was found in 76% of cases (38/50). Of those 38, polarity was correct in 81.6% (31). The main error source was implicit sentiment, such as sarcasm, metaphor and argumentation.

Interesting ideas

  • I1. Splitting the task into two independently trained taggers, where the polarity tagger takes target labels as input, lets the second stage run on gold targets or on predicted ones.
  • I2. The best tokenization granularity differs per subtask: segmenting clitics helps locate entities but hurts polarity. Keeping several segmentation views available, rather than a single canonical one, can therefore pay off.
  • I3. Distributional clusters group related entities that tend to draw the same sentiment, such as a country, its alternate name and its leader. They nearly match lexicon-plus-syntax features when rich resources are missing.
  • I4. The error analysis separates out several failure types: misleading polar words reached through English glosses, annotators skipping plausible targets, and the difficulty of ranking primary versus secondary targets.
  • I5. Lexicon polarity transferred through translation glosses misfires: "minds" glossed as "brains" is wrongly read as positive. Borrowed lexical knowledge therefore needs context checks.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): Opinion holder/target detection could be a separate pass over already-identified frame arguments, so polarity errors can be evaluated apart from argument-finding errors.
  • I2 (tokens): The en crate could expose alternate token views (e.g., split vs. unsplit possessives and contractions) so downstream modules choose the granularity that serves them best.
  • I3 (world): Cluster IDs could serve as a cheap, explainable fallback attribute on entities lacking WordNet/Wikidata links, helping group related mentions in the world model.
  • I4 (learn): Evaluation reports could tag errors by these categories and add a "primary target found" metric. This would distinguish real model failures from annotation subjectivity.
  • I5 (back): When translating through the meta-world, sentiment and connotation attributes carried over from another language should be flagged as uncertain, not copied blindly.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Columbia UniversityNew York · United States241 works · 54,053 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 works

Cite

Cite as Noura Farra and Kathy McKeown. 2017. SMARTies: Sentiment Models for Arabic Target entities. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{farra2017smarties,
  title = {{SMARTies: Sentiment Models for Arabic Target entities}},
  author = {Noura Farra and Kathy McKeown},
  booktitle = {Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers},
  year = {2017},
  doi = {10.18653/v1/e17-1094},
  url = {https://doi.org/10.18653/v1/e17-1094}
}

References

46 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Muhammad Abdul-Mageed and Mona T. Diab. 2011. Subjectivity and sentiment annotation of modern standard Arabic newswire. In Proceedings of the 5th Linguistic Annotation Workshop, pages 110– 118. Association for Computational Linguistics.
  2. Amjad Abu-Jbara, Ben King, Mona T. Diab, and Dragomir R. Radev. 2013. Identifying opinion subgroups in arabic online discussions. In ACL (2), pages 829–835.
  3. Mohammad Al-Smadi, Omar Qawasmeh, Bashar Talafha, and Muhannad Quwaider. 2015. Human annotated arabic dataset of book reviews for aspect based sentiment analysis. In Future Internet of Things and Cloud (FiCloud), 2015 3rd International Conference on, pages 726–730. IEEE. find
  4. Gilbert Badaro, Ramy Baly, Hazem Hajj, Nizar Habash, and Wassim El-Hajj. 2014. A large scale arabic sentiment lexicon for arabic opinion mining. ANLP 2014, pages 165–173.
  5. Yassine Benajiba, Mona Diab, and Paolo Rosso. 2008. Arabic named entity recognition using optimized feature sets. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 284–293. Association for Computational Linguistics.
  6. Prakhar Biyani, Cornelia Caragea, and Narayan Bhamidipati. 2015. Entity-specific sentiment classification of yahoo news comments. arXiv preprint arXiv:1506.03775. ↗
  7. Samuel Brody and Noemie Elhadad. 2010. An unsupervised aspect-sentiment model for online reviews. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 804–812. Association for Computational Linguistics.
  8. Lingjia Deng and Janyce Wiebe. 2015. Joint prediction for entity/event-level sentiment analysis using probabilistic soft logic models. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 179–189. find
  9. Li Dong, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, and Ke Xu. 2014. Adaptive recursive neural network for target-dependent twitter sentiment classification. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 49–54.
  10. Mohamed Elarnaoty, Samir AbdelRahman, and Aly Fahmy. 2012. A machine learning approach for opinion holder extraction in arabic language. arXiv preprint arXiv:1206.1011. ↗
  11. Noura Farra, Kathleen McKeown, and Nizar Habash. 2015. Annotating targets of opinions in arabic using crowdsourcing. In ANLP Workshop 2015, page 89. ↗
  12. Nizar Habash and Fatiha Sadat. 2006. Arabic preprocessing schemes for statistical machine translation. Proceedings of the Human Language Technology Conference of the NAACL, Companion Volume: Short Papers, pages 49–52.
  13. Nizar Habash, Behrang Mohit, Ossama Obeid, Kemal Oflazer, Nadi Tomeh, and Wajdi Zaghouani. 2013. QALB: Qatar Arabic language bank. In Proceedings of Qatar Annual Research Conference (ARC-2013), pages ICTP–032, Doha, Qatar. find
  14. Nizar Y. Habash. 2010. Introduction to Arabic natural language processing. Synthesis Lectures on Human Language Technologies, 3(1):1–187. ↗
  15. Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 168–177. ACM. find
  16. Ozan Irsoy and Claire Cardie. 2014. Opinion mining with deep recurrent neural networks. In EMNLP, pages 720–728. find
  17. Long Jiang, Mo Yu, Ming Zhou, Xiaohua Liu, and Tiejun Zhao. 2011. Target-dependent twitter sentiment classification. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 151–160. Association for Computational Linguistics. find
  18. Soo-Min Kim and Eduard Hovy. 2006. Extracting opinions, opinion holders, and topics expressed in online news media text. In Proceedings of the Workshop on Sentiment and Subjectivity in Text, pages 1– 8. Association for Computational Linguistics.
  19. Taku Kudo. 2005. Crf++: Yet another crf toolkit. Software available at http://crfpp.sourceforge. net. ↗
  20. John Lafferty, Andrew McCallum, and Fernando Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the eighteenth international conference on machine learning, ICML, volume 1, pages 282–289. find
  21. Pengfei Liu, Shafiq Joty, and Helen Meng. 2015. Fine-grained opinion mining with recurrent neural networks and word embeddings. In Conference on Empirical Methods in Natural Language Processing (EMNLP 2015), pages 1433–1443.
  22. James MacQueen. 1967. Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, volume 1, pages 281–297. Oakland, CA, USA. find
  23. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. ↗
  24. Margaret Mitchell, Jacqueline Aguilar, Theresa Wilson, and Benjamin Van Durme. 2013. Open domain targeted sentiment. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1643–1654.
  25. A case-study on arabic social media posts. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 767–777. find
  26. Saif M. Mohammad, Svetlana Kiritchenko, Parinaz Sobhani, Xiaodan Zhu, and Colin Cherry. 2016. Semeval-2016 task 6: Detecting stance in tweets. In Proceedings of the International Workshop on Semantic Evaluation, SemEval, volume 16, pages 31– 41.
  27. Iman Saleh, Alessandro Moschitti, Preslav Nakov, Lluís Màrquez, and Shafiq Joty. 2014. Semantic kernels for semantic parsing. In EMNLP, pages 436–442. find
  28. Ahmed Mourad and Kareem Darwish. 2013. Subjectivity and sentiment analysis of modern standard arabic and arabic microblogs. In Proceedings of the 4th workshop on computational approaches to subjectivity, sentiment and social media analysis, pages 55–64.
  29. Preslav Nakov, Alan Ritter, Sara Rosenthal, Fabrizio Sebastiani, and Veselin Stoyanov. 2016. Semeval-2016 task 4: Sentiment analysis in twitter. In Proceedings of the 10th international workshop on semantic evaluation (SemEval 2016), San Diego, US (forthcoming), pages 1–18.
  30. Islam Obaidat, Rami Mohawesh, Mahmoud Al-Ayyoub, Mohammad AL-Smadi, and Yaser Jararweh. 2015. Enhancing the determination of aspect categories and their polarities in arabic reviews using lexicon-based approaches. In Applied Electrical Engineering and Computing Technologies (AEECT), 2015 IEEE Jordan Conference on, pages 1–6. IEEE. find
  31. Arfath Pasha, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan M Roth. 2014. Madamira: A fast, comprehensive tool for morphological analysis and disambiguation of Arabic. In Proceedings of the Language Resources and Evaluation Conference (LREC), Reykjavik, Iceland, volume 14, pages 1094–1101.
  32. Maria Pontiki, Haris Papageorgiou, Dimitrios Galanis, Ion Androutsopoulos, John Pavlopoulos, and Suresh Manandhar. 2014. Semeval-2014 task 4: Aspect based sentiment analysis. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 27–35.
  33. Eshrag Refaee and Verena Rieser. 2014. Subjectivity and sentiment analysis of arabic twitter feeds with limited resources. In Workshop on Free/Open-Source Arabic Corpora and Corpora Processing Tools Workshop Programme, pages 16–21. find
  34. Josef Ruppenhofer, Swapna Somasundaran, and Janyce Wiebe. 2008. Finding the sources and targets of subjective expressions. In LREC, pages 2781–2788. find
  35. Mohammad Salameh, Saif M. Mohammad, and Svetlana Kiritchenko. 2015. Sentiment after translation: Anas Shahrour, Salam Khalifa, and Nizar Habash. 2015. Improving arabic diacritization through syntactic analysis. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1309–1315.
  36. Swapna Somasundaran and Janyce Wiebe. 2009. Recognizing stances in online debates. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 1-Volume 1, pages 226–234. Association for Computational Linguistics. find
  37. Oscar Täckström, Ryan McDonald, and Jakob Uszkoreit. 2012. Cross-lingual word clusters for direct transfer of linguistic structure. In Proceedings of the 2012 conference of the North American chapter of the association for computational linguistics: Human language technologies, pages 477–487. Association for Computational Linguistics.
  38. Duyu Tang, Bing Qin, Xiaocheng Feng, and Ting Liu. 2015. Effective lstms for target-dependent sentiment classification. arXiv preprint arXiv:1512.01100. ↗
  39. Joseph Turian, Lev Ratinov, and Yoshua Bengio. 2010. Word representations: a simple and general method for semi-supervised learning. In Proceedings of the 48th annual meeting of the association for computational linguistics, pages 384–394. Association for Computational Linguistics.
  40. Yequan Wang, Minlie Huang, xiaoyan zhu, and Li Zhao. 2016. Attention-based lstm for aspect-level sentiment classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 606–615, Austin, Texas, November. Association for Computational Linguistics.
  41. Theresa Wilson, Janyce Wiebe, and Paul Hoffmann. 2005. Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of the conference on human language technology and empirical methods in natural language processing, pages 347–354. Association for Computational Linguistics.
  42. Bishan Yang and Claire Cardie. 2013. Joint inference for fine-grained opinion extraction. In ACL (1), pages 1640–1649.
  43. Alexander Yeh. 2000. More accurate tests for the statistical significance of result differences. In Proceedings of the 18th conference on Computational linguistics-Volume 2, pages 947–953. Association for Computational Linguistics. ↗
  44. Wajdi Zaghouani, Behrang Mohit, Nizar Habash, Ossama Obeid, Nadi Tomeh, Alla Rozovskaya, Noura Farra, Sarah Alkuhlani, and Kemal Oflazer. 2014. Large scale arabic error annotation: Guidelines and framework. In LREC, pages 2362–2369. ↗
  45. Meishan Zhang, Yue Zhang, and Duy-Tin Vo. 2015. Neural networks for open domain targeted sentiment. In Proceedings of the 2015 Conference on EMNLP, pages 612–621. find
  46. Ayah Zirikly and Masato Hagiwara. 2015. Cross-lingual transfer of named entity recognizers without parallel corpora. Volume 2: Short Papers, pages 390–396. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/farra-2017-smarties-sentiment-models-arabic-target/