Elamur Research⌕
☰
home / papers / 2015 / Sentiment Classification of Arabic Documents: Experiments with multi-type features and ensemble algorithms

Sentiment Classification of Arabic Documents: Experiments with multi-type features and ensemble algorithms

Paper · 2015 · Amine Bayoudhi, Hatem Ghorbel and Lamia Hadrich Belguith

Summary

The paper addresses document-level polarity classification (positive vs. negative) for Arabic reviews and comments. The authors argue that the common recipe of n-gram features plus a single basic classifier transfers poorly between domains. Their pipeline has three phases: pre-processing, feature extraction and classification. It relies on publicly available Arabic tools and mixes several feature families: opinion features, discourse markers, stylistic, domain-dependent and morpho-lexical features. They evaluate on two widely used public corpora. OCA contains movie reviews. ACOM was collected from Aljazeera polls and forums and covers movies, sport and politics. SVM, MaxEnt and ANN are compared with ensembles built through bagging, boosting, majority voting and stacking. A majority vote over the three base learners gives the best macro-averaged F-measure, 85.06%, and beats the best single classifier on five of six datasets. An ablation shows that discourse markers help mainly for movie reviews. Because the authors use shared benchmarks, their results can be compared directly with earlier Arabic studies, and they report surpassing them.

Key points

  • OCA has 500 movie reviews (250 positive, 250 negative), labelled automatically from user rating scores. The authors note this can mislabel reviews whose text is mostly critical but whose rating is mildly positive.
  • ACOM includes DS1 (594 movie documents), DS2 (1492 comments on 18 sport topics) and a political set (1082 comments on Arab support for the Palestinian cause). It was annotated manually with four classes: positive, negative, neutral and dialectal.
  • Base classifiers:
    • MaxEnt reaches 95% F on OCA, above the earlier 93% (KNN, Mountassir et al.) and 90.73% (SVM, Rushdi-Saleh et al.).
    • SVM is the most stable learner, winning on four of six datasets.
    • Macro-averaged F is 84.58 for SVM, 84.08 for MaxEnt and 82.8 for ANN.
  • Ensembles (macro-averaged F):
    • bagging MaxEnt: 84.7
    • boosting MaxEnt: 83.38
    • majority voting over SVM, MaxEnt and ANN: 85.06
    • stacking SVM and MaxEnt with a linear-regression meta-classifier: 84.26
  • Removing discourse features lowers F by 2.4 points on OCA and 4 points on ACOM DS1. The drop is under half a point on the sport and politics sets, and there is no change on ACOMA and ACOMB.

Interesting ideas

  • I1. Instead of reporting one global number for a feature family, the authors measure its contribution separately for each domain, which shows where it actually helps.
  • I2. A plain majority vote over learners of different types beat both the individual models and more elaborate schemes like boosting and stacking.
  • I3. With no Arabic discourse parser available, surface discourse markers served as a lightweight stand-in for rhetorical structure and still brought gains.
  • I4. The paper warns that labels derived automatically from star ratings can contradict the actual text.
  • I5. The authors trace dataset difficulty to topical coherence: many sports in one set was hard, while one political issue was easy, regardless of domain breadth.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Ablate each component (e.g., a repair rule or a frame feature) per genre, such as fables, maths problems and school texts, so its value is not hidden inside an average.
  • I2 (tree): Arc-level majority voting between the perceptron parser and gparse (plus another witness) is a cheap, explainable way to get trees that beat each parser alone.
  • I3 (frames): Connectives such as "but" or "although" can mark contrast between clauses, helping prag decide which clause carries the speaker's real stance and flag possible irony or implicature.
  • I4 (learn): Any silver label inferred from an external signal should be checked against the text itself, and disagreements routed to an LLM teacher tier rather than trusted blindly.
  • I5 (answer): Report QA and test scores alongside a measure of topical homogeneity, so a drop in accuracy can be told apart from a genuinely harder, more heterogeneous test set.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of SfaxSfax · Tunisia14 works · 231 citations

Also on the byline: HES-SO University of Applied Sciences and Arts Western Switzerland.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Sentiment & emotion · 972 worksSemitic languages · 2,908 works

Cite

Cite as Amine Bayoudhi, Hatem Ghorbel and Lamia Hadrich Belguith. 2015. Sentiment Classification of Arabic Documents: Experiments with multi-type features and ensemble algorithms. Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{bayoudhi2015sentiment,
  title = {{Sentiment Classification of Arabic Documents: Experiments with multi-type features and ensemble algorithms}},
  author = {Amine Bayoudhi, Hatem Ghorbel and Lamia Hadrich Belguith},
  booktitle = {Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation},
  year = {2015},
  url = {https://aclanthology.org/Y15-1023/}
}

References

44 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abbasi A., Chen H., Salem A. 2008. Sentiment analysis in multiple languages: Feature selection or opinion classification in Web forums. ACM Transactions on Information Systems 26(3):12. find
  2. Abdul-Mageed M., Diab M. 2012. AWATIF: A Multi-Genre Corpus for Modern Standard Arabic Subjectivity and Sentiment Analysis. In Proc LREC 2012, pp. 3907-3914.
  3. Ahmad K., Cheng D., Almas Y. 2006. Multi-lingual sentiment analysis of financial news streams. In Proc. of the 1st International Conference on Grid in Finance. find
  4. Almas Y., Ahmad K. 2007. A note on extracting sentiments in financial news in English, Arabic & Urdu. In Proc of Workshop on Computational Approaches to Arabic Script-based Languages. find
  5. Al-Saif A., Markert K. 2010. The Leeds Arabic discourse treebank: Annotating discourse connectives for Arabic. In Proc of LREC, 17-23 May, Malta.
  6. Al-Twairesh N., Al-Khalifa H., Al-Salman A. 2014. Subjectivity and Sentiment Analysis of Arabic: Trends and Challenges. In Proc. Of the 11th ACS/IEEE International Conference on Computer Systems and Applications (AICCSA' 2014), Doha, Qatar, 10-13 November. find
  7. Asher N. 1993. Reference to Abstract Objects in Discourse. Kluwer Dordrecht. ↗
  8. Bayoudhi A., Koubaa H., Ghorbel H., Hadrich Belguith L. 2014. Vers un lexique arabe pour l’analyse des opinions et des sentiments, In Proc of the 5th International Conference on Arabic Language Processing CITALA’14, Oujda, Morocco, 26 – 27 November. find
  9. Boudabous M.M., Chaâben Kammoun N., Khedher N., Hadrich Belguith L., Sadat F. 2013. Arabic Word-Net semantic relations enrichment through morpho-lexical patterns, in Proc. of the First International Conference on Communications, Signal Processing, and their Applications (ICCSPA’13), Sharjah, UAE, February 12-14. find
  10. Breiman L. 1994. Bagging predictors. Technical Report 421, Department of Statistics, University of California, Berkeley.
  11. Chardon B. 2013. Chaîne de traitement pour une approche discursive de l'analyse d'opinion. Phd dissertion UPS France. find
  12. Elarnaoty M., AbdelRahman S., Fahmy A. 2012. A machine learning approach for opinion holder extraction in Arabic, International Journal of Artificial Intelligence & Applications; March 2012, Vol. 3 Issue 2, p45. find
  13. El-Halees A. 2011. Arabic Opinion Mining Using Combined Classification Approach. In Proc of the international Arab Conference on Information Technology, 11-14 December, Riyadh, Saudi Arabia. find
  14. Esuli A., Sebastiani F. 2006. SentiWordNet: A Publicly Available Lexical Resource for Opinion Mining. In Proc of the fifth international conference on Language Resources and Evaluation, 22-28 May, Genoa, Italy.
  15. Hall M., Frank E., Holmes G., Pfahringer B., Reutemann P., Witten I.H. 2009. The WEKA data mining software: an update, ACM SIGKDD Explorations Newsletter, Volume 11 Issue 1, pp 10-18, June 2009. find
  16. Hu M., Liu B. 2004. Mining Opinions Features in Customer Reviews. Proc of 19 th international conference on Artificial Intelligence AAAI, 25-29 July, San Jose, California, pp. 755-760. find
  17. Keskes I. 2015. Discourse Analysis of Arabic Documents and Application to Automatic Summarization. PhD dissertion, UPS France. ↗
  18. Khoja S. and Garside R. 1999. Stemming Arabic Text, UK: Computing Department, Lancaster University. find
  19. Khalifa I., Feki Z., Farawila A. 2011. Arabic discourse segmentation based on rhetorical methods. In Electric Computer Sciences 11. find
  20. Korayem M., Crandall D., Abdul-Mageed M. 2012. Subjectivity and Sentiment Analysis of Arabic: A Survey. in Advanced Machine Learning Technologies and Applications AMLTA, 322. find
  21. Kuncheva L. 2004. Combining pattern classifiers: methods and algorithms. John Wiley & Sons. find
  22. Li S., Xia R., Zong C., Huang C.-R. 2009. A framework of feature selection methods for text categorization. In Proc of the 47 th Annual Meeting of the Association for Computational Linguistics, 2-7 August, suntec, Singapore. find
  23. Liu B. 2009. Sentiment Analysis and Opinion Mining. Boca Raton: Morgan & Claypool Publishers. find
  24. Liu B. 2011. Web data mining: Exploring hyperlinks, New York: Springer, Lovins, J. B. (1968). Development of a stemming algorithm. Mechanical Translation and Computational Linguistic. find
  25. Liu H., Motoda H. 2008. Computational Methods of Feature Selection, Boca Raton: Chapman & Hall. find
  26. Manning C.D., Raghavan P., Schtze H. 2008. Introduction to information retrieval. Cambridge University Press. ↗
  27. Maynard D., Funk A. 2011. Automatic detection of political opinions in tweets. In Proc of the 10 th International semantic web conference, 23-27 October, Bonn, Germany. Medhat W., Hassan A., Korashy A. 20140. Sentiment analysis algorithms and applications: A survey. Ain Shams Engineering Journal 5, pp. 1093–1113. find
  28. Mejova Y., Srinivasan P. 2011. Exploring Feature Definition and Selection for Sentiment Classifiers. In Proc of the Fifth International AAAI Conference on Weblogs and Social Media ICWSM, 17-21 July, Barcelona, Spain. find
  29. Monroe W., Green S., Manning C.D. 2014. Word Segmentation of Informal Arabic with Domain Adaptation. In Proc of the 52 nd Annual Meeting of the Association for Computational Linguistics, 22-27 June, Baltimore, USA.
  30. Moraes R., Francisco Valiati J., Gavião Neto W.P. 2013. Document-level sentiment classification: An empirical comparison between SVM and ANN. Expert Systems with Applications 40, pp. 621-633. ↗
  31. Mountassir A., Benbrahim H., Berrada I. 2013. Sentiment classification on Arabic corpora: A preliminary cross-study. Document Numérique 16(1): 73-96. ↗
  32. Nakagawa T., Inui K., Kurohashi S. 2010. Dependency Tree-based Sentiment Classification using CRFs with Hidden Variables, Human Language Technologies, The 2010 Annual Conference of the North American Chapter of the ACL NAACL HLT, pp 786–794, 1-6 June, Los Angeles, California.
  33. Palmer F. 1986. Mood and Modality. Cambridge University Press. find
  34. Pang B., Lee L., Vaithyanathan S. 2002. Thumbs up? Sentiment Classification using Machine Learning Techniques, In Proc of the Conference on Empirical Methods in Natural Language Processing EMNLP, 6-7 July, Philadelphia, PA, USA.
  35. Pasha A., Al-Badrashiny M., Diab M.T., El-Kholy A., Eskander R., Habash N., Pooleery M., Rambow O., Roth R. 2014. MADAMIRA: A Fast, Comprehensive Tool for Morphological Analysis and Disambiguation of Arabic. In Proc of the The 9 th edition of the Language Resources and Evaluation Conference LREC, 26-31 May, Reykjavik, Iceland.
  36. Rushdi-Saleh M., Martın-Valdivia M., Urena-Lopez L., Perea-Ortega J. 2011. OCA: Opinion corpus for Arabic. Journal of the American Society for Information Science and Technology 62(10). ↗
  37. Saurı R. 2008. A Factuality Profiler for Eventualities in Text. PhD dissertation. find
  38. Schapire R. E., Freund Y., Bartlett P., Lee W.S. 1998. Boosting the margin: A new explanation for the effectiveness of voting methods. Annals of Statistics. ↗
  39. Somasundaran S., Namata G., Wiebe J., Getoor L. 2009. Supervised and unsupervised methods in employing discourse relations for improving opinion polarity classification. In Proc of the conference on Empirical Methods in Natural Language Processing EMNLP, 6-7 August, Suntec, Singapore.
  40. Syarif I, Zaluska E., Prugel-Bennett A., Wills G. 2012. Application of bagging, boosting and stacking to intrusion detection. In Proc of the 8 th International Conference on Machine Learning and Data Mining MLDM, 13-20 Jul, Berlin, DE. find
  41. Taboada M., Brooke J, Tofiloski M., Voll K., Stede M. 2011. Lexicon-based methods for sentiment analysis. Computational Linguistics 37, pp 267 -307. ↗
  42. Van Asch V. 2013. Macro- and micro-averaged evaluation measures [[basic draft]]. find
  43. Wang G., Sun J., Ma J., Xue K., Gud J. 2014. Sentiment classification: The contribution of ensemble learning. Decision Support Systems 57, pp. 77–93. ↗
  44. Xia R., Zong C., Li S. 2011. Ensemble of feature sets and classification algorithms for sentiment classification, Information Sciences 181, pp 1138–1152. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/bayoudhi-2015-sentiment-classification-arabic-documents-experiments/