Elamur Research⌕
☰
home / papers / 2021 / Sentiment Analysis in Poems in Misurata Sub-dialect -- A Sentiment Detection in an Arabic Sub-dialect

Sentiment Analysis in Poems in Misurata Sub-dialect -- A Sentiment Detection in an Arabic Sub-dialect

Summary

The study asks whether sentiment can be detected automatically in poetry composed in the Misurata sub-dialect of Libyan Arabic. This variety had no prior sentiment work, and earlier Arabic poetry research looked only at MSA. Poems came from four local poets. An expert in Arabic literature and rhetoric helped assign sentiment labels. The author compared two approaches:

  • Traditional classifiers in Sklearn: Logistic Regression, Random Forest, Naive Bayes and SVM, evaluated with 10-fold cross-validation because the dataset is small.
  • Mazajak: a web-based CNN sentiment tool built for MSA and major Arabic dialects. Training lines were submitted one sentence at a time with feedback corrections, and the test portion was scored in batch mode.

The classic models reached about 68% accuracy and Mazajak about 61%. This goes against the common claim in the ALSA literature that deep models beat conventional ones. The author attributes the gap partly to poetry's dense figurative language and to the tool's lack of exposure to sub-dialects. The work matters as a first probe of a low-resource sub-dialect and of multi-line literary text, rather than isolated social-media posts.

Key points

  • Data: the corpus is made up of Misurata Arabic poems written by four poets, namely Salah Abaid and Khalid Iqsheerah, along with Mohamed Emmiama and Alhusian Elasawadi. Sentiment labelling was supported by a specialist in Arabic literature and rhetoric.
  • Sklearn accuracies: Logistic Regression 67%, Random Forest 68%, Naive Bayes 69% (best), SVM 68%.
  • Mazajak accuracies: 60.93% on training data and 60.66% on test data, so the feedback-based adaptation gave little gain.
  • Overall comparison: traditional ML averaged 68.0% versus 60.66% for the CNN tool.
  • Cited prior finding: Mazajak performs markedly worse on sentences containing metaphors (Alsiyat and Piao).

Interesting ideas

  • I1. On a tiny, out-of-domain dataset, plain linear and probabilistic classifiers beat a pretrained deep tool, so cheap baselines should always be run before trusting a bigger model. — For Mova (learn): every new learned component should be scored against a trivial baseline under the same cross-validation, as a built-in control against overclaiming.
  • I2. Comparing a tool's accuracy on the data it was "taught" with its accuracy on held-out data shows whether user-feedback adaptation actually changed anything. Near-identical scores reveal that it did not. — For Mova (learn): self-induced repair rules and teacher corrections could be checked the same way, measuring before/after on corrected items versus unseen ones to confirm the repair generalizes.
  • I3. The paper notes that poem lines depend on surrounding lines and the author's overall theme, so judging each sentence alone discards the document-level signal. — For Mova (frames): sentiment, irony and implicature judgments in prag should draw on discourse context and coreference across sentences, not just the current clause.
  • I4. Figurative devices such as metaphor, personification and hyperbole are named as a likely cause of failure, which suggests detecting figurative use before assigning polarity. — For Mova (frames): a metaphor or hyperbole flag on a frame could block literal sentiment and role inference and route the clause to the irony and implicature logic.
  • I5. For an under-documented language variety, labels were grounded by a domain expert rather than crowd annotators, which buys reliability where no lexicons exist. — For Mova (learn): when silver data from witness parsers disagree on rare or dialectal constructions, a small expert-labelled set can serve as the gold arbiter for evaluation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Sentiment & emotion · Semitic languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2109.07203 · arXiv 2109.07203

@misc{abugharsa2021sentiment,
  title = {{Sentiment Analysis in Poems in Misurata Sub-dialect -- A Sentiment Detection in an Arabic Sub-dialect}},
  author = {Abugharsa, Azza},
  year = {2021},
  doi = {10.48550/arxiv.2109.07203},
  eprint = {2109.07203},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2109.07203}
}

References

63 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abdul-Mageed M., Kuebler S., and Diab M. (2012). AWATIF: a multi-genre corpus for modern standard Arabic subjectivity and sentiment analysis [Paper presentation]. LREC Conference, Istanbul, Turkey. find
  2. Abdul-Mageed, M., Diab, M., and Kubler, S. (2014). Samar: Subjectivity and sentiment analysis for Arabic social media. Computer Speech and Language, 28(1), 20-37. find
  3. Abdulla, N., Ahmed, N., Shehab, M., and Al-Ayyoub, M. (2013). Arabic sentiment analysis: Lexicon-based and corpus-based [Paper presentation]. AEECT Conference, NJ, USA. find
  4. Abu Farha, I. and Magdy, W. (2019). An online Arabic sentiment analyser. Association for Computational Linguistics, 192-198. find
  5. Abu Farha, I. and Magdy, W. (2019). Mazajak. [Computer software]. The University of Edinburgh. http://mazajak.inf.ed.ac.uk:8000/ ↗
  6. Abu Kwaik, K., Chatzikyriakidis, S., Dobnik, S., Saad, M., and Johansson, R. (2020). An Arabic tweets sentiment analysis dataset (ATSAD) using distant supervision and self training. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools with a Shared Task on Offensive Language Detection, 1-8. find
  7. Ahmad, M., Aftab, S., Bashir, M., Hameed, N. (2018). Sentiment analysis using SVM: A systematic literature review. International Journal of Advanced Computer Science and Applications, 9(2), 182-188. find
  8. Ahmed, M., Hasan, R., Ali, A., and Mohammed, M. (2019). The classification of the modern Arabic poetry using machine learning. TELKOMNIKA, 17(5), 2667-2674. 9. find
  9. Ain, Q., Ali, A., Riaz, A., Noureen, A., Kamran, M., Hayat, B., and Rehman, A. (2017). Sentiment analysis using deep learning techniques: A review. International Journal of Advanced Computer Science and Applications, 8(6), 424-433. 10. find
  10. Alayaba, A., Palade, V., England, M., and Iqbal, R. (2018). A combined CNN and LSTM model for Arabic sentiment analysis. Springer International Publishing, 11015, 179-191. 11. find
  11. Albayati, A., Al-Araji, A., Ameen, S. (2020). Arabic sentiment analysis (ASA) using deep learning approach. Journal of Engineering, 6(26), 85-93. 12. find
  12. Ali, M. (2021). Arabic sentiment analysis about online learning to mitigate covid-19. Journal of Intelligent Systems, 30(1), 524-540. 13. find
  13. Al-Amrani, Y., Lazaar, M., Eddine, K., and Kadiri, E. (2018). Sentiment analysis using a hybrid method of support vector machine and decision tree. Journal of Theoretical and Applied Information Technology, 96(7). 1886-1895. 14. find
  14. Al-Ayyoub, M., Bani Essa, S., and Alsmadi, I. (2015). Lexicon-based sentiment analysis of Arabic tweets. International Journal of Social Network Mining, 2(2), 101-114. 15. find
  15. Al-Ayyoub, M., Khamaiseh, A., Jararweh, Y. and Al-Kabi, M. (2019). A comprehensive survey of Arabic sentiment analysis. Information Processing and Management, 56(2), 320-342. 16. find
  16. Al-Azani, S. and El-Alfy, E. (2017). Hybrid deep learning for sentiment polarity determination of Arabic microblogs. Springer International Publishing, 10638, 491-500. 17. find
  17. Al-Balushi, R. (2012). Why verbless sentences in Standard Arabic are verbless. The Canadian Journal of Linguistics, 57,1-30. 18. find
  18. Al-Humoud, S., Altuwaijri, M., Albuhairi, T., Alohaideb, W. (2015). Survey on Arabic sentiment analysis in Twitter. International Journal of Social, Behavioral, Educational, Economic and Management Engineering, 9, 364-368. 19. find
  19. Alhumoud, S., Albuhairi, T. and Altuwaijri, M. (2015). Arabic sentiment analysis using WEKA a hybrid learning approach. Knowledge Discovery, Knowledge Engineering and Knowledge Management,1, 402-408. 20. find
  20. Almuqren, L., Alzammam, A., Alotaibi, S., Cristea, A. and Alhumoud, S. (2017). A review on corpus annotation for Arabic sentiment analysis. Springer, 215-225. 21. find
  21. Al-Radaideh Q., and Al-Qudah G. (2017). Application of rough set-based feature selection for Arabic sentiment analysis. Cogn. Comput, 9, 436-45. 22. find
  22. Alsharif, O., Alshamaa, D., and Ghneim, N. (2013). Emotion classification in Arabic poetry using Machine Learning. International Journal of Computer Applications, 65(16),10-15. 23. find
  23. Alshutayri, A., Atwell, E., AlOsaimy, A., Dickins, J., Ingleby, M., and Watson, J. (2016). Arabic language WEKA-based dialect classifier for Arabic automatic speech recognition transcripts. Varieties and Dialects Conference. Osaka, Japan. 24. find
  24. Al-Samdi, M., Qawasmeh, O., Al-Ayyoub, M., Jararweh, Y., and Gupta, B. (2017). Deep recurrent neural network vs. support vector machine for aspect-based sentiment analysis of Arabic hotels’ reviews. Journal of Computational Science, 27, 386-393. 25. find
  25. Al-Sallab, A., Baly, R. Hajj, H., Shaban, K., El-Hajj, W., Badaro, G. (2017). AROMA: A recursive deep learning model for opinion mining in Arabic as a low resource language. ACM Journal, 16(4), 1-20. 26. find
  26. Alsayat, A. and Elmitwally, N. (2020). A comprehensive study for Arabic sentiment analysis (challenges and applications). Egyptian Informatics Journal, 7-12. 27. find
  27. Alsiyat, I. and Piao, S. (2020). Metaphorical expressions in automatic Arabic sentiment analysis. Lancaster University, 1-6. 28. find our card
  28. Antoun, W. Baly, F. and Hajj, H. (2020). AraBERT: Transformer-based model for Arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, 9-15. 29. find our card
  29. Baali, M., and Ghneim, N. (2019). Emotion analysis of Arabic tweets using deep learning approach. Journal of Big Data, 6(8), 7-12. 30. find
  30. Baly, R., Badaro, G., El-Khoury, G., Moukalled, R., Aoun, R., Hajj, H. (2017). A characterization study of Arabic Twitter data with a benchmarking for state-of-the-art opinion mining models. Association for Computational Linguistics, 110-118. 31. find
  31. Boudad, N., Faizi, R., Thami, R., and Chiheb, R. (2018). Sentiment analysis in Arabic: A review of the literature. Ain Shams Engineering Journal, 9(4), 2479-2490. 32. find
  32. Darwish, K., Magdy, W. (2014). Arabic information retrieval. Foundations and Trends® in Information Retrieval, 7(4), 239-342. 33. find
  33. Devi, B., Bai, V., Ramasubbareddy, S., Govinda, K. (2020). Emerging Research in Data Engineering Systems and Computer Communications: Sentiment Analysis on Movie Reviews. P. Venkata Krishna and Mohammad Obaidat. 34. find
  34. Elarnaoty, M. (2012). A machine learning approach for opinion holder extraction in Arabic language. International Journal of Artificial Intelligence, 3, 45-63. 35. find
  35. Elfaik, H. and Nfaoui, E. (2020). Deep bidirectional LSTM network learning-based sentiment analysis for Arabic text. Journal of Intelligent Systems, 30, 395-412. 36. find
  36. El-Halees, A. (2017). Arabic opinion mining using distributed representations of documents. In Proceedings of the 2017 Palestinian International Conference on Information and Communication Technology, 28-33. 37. find
  37. Elmasry, M., Soliman, T., and Hedar, A. (2014). Sentiment analysis of Arabic slang comments on Facebook. International Journal of Computers and Technology, 12(5), 3470-3478. 38. find
  38. Elramli, Y. (2012). Assimilation in the Phonology of a Libyan Arabic Dialect: A Constraint-based approach. Ph.D. dissertation. Newcastle University. 39. find
  39. Galal, M., Madbouly, M., and El-Zoghby, A. (2019). Classifying Arabic text using deep learning. Journal of Theoretical and Applied Information Technology, 97(23), 3412-3422. 40. find
  40. Ghallab, A., Mohsen, A., and Ali, Y. (2020). Arabic sentiment analysis: A systematic literature review. find
  41. Applied Computational Intelligence and So Computing, 2020, 1-21. 41. find
  42. Habash, N. (2010). Introduction to Arabic natural language processing. Synthesis Lectures on Human Language Technologies, 3(1), 1-187. 42. find
  43. Hammam, M., Elmahdy, A., Halawa, A., and Youness, H. (2018). Improve the automatic classification accuracy for Arabic tweets using ensemble methods. Journal of Electrical Systems and Information Technology, 5(3), 363-370. 43. find
  44. Heikal, M., Torki, M., and El-Makky, N. (2018). Sentiment analysis of Arabic Tweets using deep learning. Procedia Computer Science, 142(1), 114-122. 44. find
  45. Hemmatian, F., and Sohrabi, M. (2017). A survey on classification techniques for opinion mining and sentiment analysis. Springer, 52(1), 1495-1545. 45. find
  46. Ibrahim, H., Abdou, S., and Gheith, M. (2015). Sentiment analysis for modern standard Arabic and colloquial. International Journal on Natural Language Computing,4(2), 95-109. 46. Jin, H., Zhu,Y., Jin, Z. and Arora, S. (2014). Sentiment visualization on Tweets stream. Journal of Software, 9(9), 2348-2352. 47. find
  47. Khasawneh, R. Wahsheh, H. Al Kabi M. and Aismadi, I. (2013) Sentiment analysis of Arabic social media content: A comparative study. Proceedings of the 8th International Conference for Internet Technology and Secured Transactions (ICITST), 101-106. 48. find
  48. Liu, B. (2012). Sentiment analysis and Opinion Mining. Synthesis Lectures on Human Language Technologies, 5(1), 1-167. 49. find
  49. Maiteq, T. (2020). A Phonetic study of vowel epenthesis in the dialect of Misurata: Preliminary results. Journal of Academic Research (Humanities and Social Science), 16, 1-6. 50. find
  50. Mamtesh, M. and Mehla, S. (2019). Sentiment analysis of movie reviews using machine learning classifiers. International Journal of Computer Applications, 182(50), 25-28. 51. find
  51. Moussa, M., Mohamed, E., Haggag, M. (2018). A generic lexicon-based framework for sentiment analysis. International Journal Computers and Applications, 42(5), 1-11. 52. find
  52. Nabil, M., Aly, M. and Atiya, A. (2015). LABR: A large-scale Arabic sentiment analysis benchmark. Cornell University. 1-9. 53. find
  53. Oraby, S., El-Sonbaty, Y., and Abou El-Nasr, M. (2013). Finding opinion strength using rule-based parsing for Arabic sentiment analysis. Springer, 509–520. 54. find
  54. Pang, B., Lee, L., and Vaithyanathan, S. (2002). Thumbs up?: sentiment classification using machine learning techniques. In Proceedings of the ACL-02 conference on Empirical methods in natural language processing, 10, 79-86. Association for Computational Linguistics. 55. find our card
  55. Pang, B. and Lee, L. (2008). Opinion mining and sentiment analysis. Foundation and Trends in Information Retrieval, 2(1-2), 1-135. 56. find
  56. Refaee, E., and Rieser, V. (2014). An Arabic Twitter corpus for subjectivity and sentiment analysis. Association for Computational Linguistics, 2268-2273. 57. find
  57. Rosenthal, S., Farra, N., and Nakov, P. (2017). Semeval-2017 task 4: Sentiment analysis in Twitter. Association for Computational Linguistics, 502-518. 58. find our card
  58. Shoeb, M., and Ahmed, J. (2017). Sentiment analysis and classification of tweets using data mining. International Research Journal of Engineering and Technology, 4(12), 1471-1474. 59. find
  59. Shoukry, A. and Rafea, A. (2012). Sentence level Arabic sentiment analysis. Proceedings of the International Conference on Collaboration Technologies and Systems. Denver, USA. 546-550. 60. find
  60. Theelwall, M., Buckley, K., Paltoglou, G., Cai, D., and Kappas, A. (2010). Sentiment strength detection in short informal text. Journal of the American Society for Information Science and Technology, 61(12), 2544-2558. 61. find
  61. The Encyclopedia Britannica (2020). Retrieved April 17, 2021, from https://www.britannica.com/place/Misurata 62. ↗
  62. WEKA. (2014) WEKA 3: Data Mining Software in Java, [Computer software] The University of Waikato. https://www.cs.waikato.ac.nz/ml/weka/ 63. ↗
  63. Zahidi, Y., Younoussi, Y. and Al-Amrani, Y. (2020). A powerful comparison of deep learning frameworks for Arabic sentiment analysis. International Journal of Electrical and Computer Engineering (IJECE), 11(1), 745-752. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/abugharsa-2021-sentiment-analysis-poems-misurata-sub/