Elamur Research⌕
☰
home / papers / 2019 / A Comparative Study of Feature Selection Methods for Dialectal Arabic Sentiment Classification Using Support Vector Machine

A Comparative Study of Feature Selection Methods for Dialectal Arabic Sentiment Classification Using Support Vector Machine

Summary

Sentiment classification of Arabic is difficult because the language is morphologically rich and because dialectal text follows no fixed spelling or grammar conventions. N-gram representations also produce very high-dimensional feature vectors. Feature selection is well studied for English but has received little attention for dialectal Arabic, and combinations of selectors have rarely been tried. Al-Harbi uses a balanced set of 2400 reviews (1200 positive, 1200 negative) in MSA and colloquial Jordanian, drawn from an existing dataset and spanning domains such as restaurants, hotels, education and tourism. He compares five selectors with an SVM classifier in Rapidminer: Information Gain, Correlation, SVM-based selection, Gini Index and Chi-Square. He also varies weighting schemes, stemmers, stop-word removal and n-gram models. Among single methods, SVM-based selection performs best. Chaining Correlation and then SVM selection on uni-grams gives the highest accuracy, 93.25%. The study shows that stacking selectors can beat any single one on noisy dialectal data.

Key points

  • The data are 2400 balanced binary-polarity reviews mixing MSA and Jordanian dialect across eight domains. Results come from cross-validation.
  • Preprocessing removes noise (repeated letters, diacritics, elongation, numerals, English words) and normalizes letter variants, mapping alef forms to ا, ى/ئ to ي, ة to ه and ؤ to و.
  • The best individual selectors are:
    • SVM-based selection: 92.12% accuracy with the top 4000 features.
    • Correlation: 91.29% with the top 3500.
    • Chi-Square: peaks at 88.63%.
  • In the combined experiments, Correlation (3500) followed by SVM (1500) reaches 93.25% accuracy and 94.48% precision. The reverse order, SVM (4000) followed by Correlation (1500), reaches 92.42%.
  • The best text representation was uni-grams with TF-IDF weighting and stop-word removal.

Interesting ideas

  • I1. Selectors are chained in sequence: a broad filter keeps a large pool, then a model-aware selector picks a smaller subset from it, and the order of the two steps changes the outcome.
  • I2. The author notes that SVM-based selection may score well with an SVM classifier simply because both share one prediction model, which is a warning about circular evaluation.
  • I3. The size of the second-stage subset is searched around the best value found in the previous experiment instead of over a full grid.
  • I4. Spelling variants of letters are collapsed before tokenization so that dialectal writing differences do not split one word into many forms.
  • I5. Balancing classes by random subsampling keeps accuracy figures from being inflated by the majority class.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Self-induced repair rules could be pruned in the same two stages, first a cheap frequency or agreement filter, then a model-based check, with both orders compared.
  • I2 (learn): When a parser both filters silver data and is scored on it, an independent witness or negative control should check that the gain is not self-confirmation.
  • I3 (learn): Thresholds such as how many witness parsers must agree could be tuned incrementally, each stage anchored at the previous optimum, which saves compute.
  • I4 (tokens): A comparable normalization layer for English variants (British/American spellings, elongated "soooo", repeated punctuation) could map forms to canonical lemmas while keeping the original surface form for explainability.
  • I5 (answer): Test sets for school-course questions or fable QA could be balanced across answer types so reported scores reflect real understanding rather than skew.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics Sentiment & emotion Semitic languages

 Institutions

🎓Jazan UniversityGazan · Saudi Arabia4 works · 60 citations

Cite

Cite as Omar Al-Harbi. 2019. A Comparative Study of Feature Selection Methods for Dialectal Arabic Sentiment Classification Using Support Vector Machine. arXiv:1902.06242 · IJCSNS International Journal of Computer Science and Network Security, VOL.19 No.1, January 2019, 167-176.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1902.06242 · arXiv 1902.06242

@misc{alharbi2019comparative,
  title = {{A Comparative Study of Feature Selection Methods for Dialectal Arabic Sentiment Classification Using Support Vector Machine}},
  author = {Omar Al-Harbi},
  year = {2019},
  doi = {10.48550/arxiv.1902.06242},
  eprint = {1902.06242},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1902.06242}
}

References

44 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Liu, B., “Sentiment Analysis and Subjectivity”. Handbook of natural language processing, 2010. 2: p. 627-666. find
  2. Al‐Sughaiyer, I.A. and I.A. Al‐Kharashi, “Arabic morphological analysis techniques: A comprehensive survey”. Journal of the American Society for Information Science and Technology, 2004. 55(3): p. 189-213. find
  3. El-Beltagy, S.R. and A. Ali, “Open issues in the sentiment analysis of Arabic social media: A case study”. in 2013 9th International Conference on Innovations in Information Technology (IIT). 2013. IEEE. find
  4. Harrington, P., “Machine learning in action”. Shelter Island, NY: Manning Publications Co, 2012. find
  5. Pang, B., L. Lee, and S. Vaithyanathan, “Thumbs up?: sentiment classification using machine learning techniques”. in Proceedings of the ACL-02 conference on Empirical methods in natural language processing-Volume 10. 2002. Association for Computational Linguistics. our card
  6. Agarwal, B. and N. Mittal, “Prominent feature extraction for review analysis: an empirical study”. Journal of Experimental & Theoretical Artificial Intelligence, 2016. 28(3): p. 485-498. find
  7. Abbasi, A., H. Chen, and A. Salem, “Sentiment analysis in multiple languages: Feature selection for opinion classification in Web forums”. ACM Trans. Inform. Syst, 2008. 26(3): p. 12. find
  8. Ibrahim, H.S., S.M. Abdou, and M. Gheith, “Sentiment analysis for modern standard Arabic and colloquial”. International Journal on Natural Language Computing (IJNLC), 2015. 4(2): p. 95-109. find
  9. Shoukry, A. and A. Rafea, “A hybrid approach for sentiment classification of Egyptian Dialect Tweets”. in 2015 First International Conference on Arabic Computational Linguistics (ACLing). 2015. IEEE. find
  10. Aldayel, H.K. and A. Azmi, “Arabic tweets sentiment analysis–a hybrid scheme”. Journal of Information Science, 2016. 42(6): p. 782-797. find
  11. Al-Harbi, O., “Using objective words in the reviews to improve the colloquial arabic sentiment analysis”. International Journal on Natural Language Computing (IJNLC), 2017. 6(3): p. 1-14. our card
  12. Guyon, I. and A. Elisseeff, “An introduction to variable and feature selection”. Journal of machine learning research, 2003. 3(Mar): p. 1157-1182. find
  13. Gamon, M., “Sentiment classification on customer feedback data: noisy data, large feature vectors, and the role of linguistic analysis”. in Proceedings of the 20th international conference on Computational Linguistics. 2004. Association for Computational Linguistics. find
  14. Kennedy, A. and D. Inkpen, “Sentiment classification of movie reviews using contextual valence shifters”. Computational intelligence, 2006. 22(2): p. 110-125. find
  15. Agarwal, A., et al., “Sentiment analysis of twitter data”. in Proceedings of the workshop on languages in social media. 2011. Association for Computational Linguistics. our card
  16. Forman, G., “An extensive empirical study of feature selection metrics for text classification”. Journal of machine learning research, 2003. 3(Mar): p. 1289-1305. find
  17. Yu, L. and H. Liu, “Efficient feature selection via analysis of relevance and redundancy”. Journal of machine learning research, 2004. 5(Oct): p. 1205-1224. find
  18. Agarwal, B. and N. Mittal, “Sentiment classification using rough set based hybrid feature selection”. in Proceedings of the 4th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis. 2013. find
  19. Tang, J., S. Alelyani, and H. Liu, “Feature selection for classification: A review”. Data classification: Algorithms and applications, 2014: p. 37. find
  20. Liu, H. and H. Motoda, Feature selection for knowledge discovery and data mining. Vol. 454. 2012: Springer Science & Business Media. find
  21. Pang, B. and L. Lee, “Opinion mining and sentiment analysis”. Foundations and Trends® in Information Retrieval, 2008. 2(1–2): p. 1-135. find
  22. Lin, Y., et al., “An information theoretic approach to sentiment polarity classification”. in Proceedings of the 2nd joint WICOW/AIRWeb workshop on web quality. 2012. ACM. find
  23. Moraes, R., J.F. Valiati, and W.P.G. Neto, “Document-level sentiment classification: An empirical comparison between SVM and ANN”. Expert Systems with Applications, 2013. 40(2): p. 621-633. find
  24. Deng, Z.-H., K.-H. Luo, and H.-L. Yu, “A study of supervised term weighting scheme for sentiment analysis”. Expert Systems with Applications, 2014. 41(7): p. 3506-3513. find
  25. O’Keefe, T. and I. Koprinska, “Feature selection and weighting methods in sentiment analysis”. in Proceedings of the 14th Australasian document computing symposium, Sydney. 2009. Citeseer. find
  26. Wang, S., et al., “A feature selection method based on fisher’s discriminant ratio for text sentiment classification”. in International Conference on Web Information Systems and Mining. 2009. Springer. find
  27. Nicholls, C. and F. Song, “Comparison of feature selection methods for sentiment analysis”. in Canadian Conference on Artificial Intelligence. 2010. Springer. find
  28. Agarwal, B. and N. Mittal, “Categorical probability proportion difference (CPPD): a feature selection method for sentiment classification”. in Proceedings of the 2nd workshop on sentiment analysis where AI meets psychology, COLING. 2012. find
  29. Duwairi, R. and M. El-Orfali, “A study of the effects of preprocessing strategies on sentiment analysis for Arabic text”. Journal of Information Science, 2014. 40(4): p. 501-513. find
  30. Khalil, T., et al., “Which configuration works best? an experimental study on supervised Arabic twitter sentiment analysis”. in 2015 First International Conference on Arabic Computational Linguistics (ACLing). 2015. IEEE. find
  31. Aliane, A., et al., “A genetic algorithm feature selection based approach for Arabic Sentiment Classification”. in IEEE/ACS 13th International Conference of Computer Systems and Applications (AICCSA). 2016. IEEE. find
  32. El-Naggar, N., Y. El-Sonbaty, and M.A. El-Nasr, “Sentiment analysis of modern standard Arabic and Egyptian dialectal Arabic tweets”. in 2017 Computing Conference. 2017. IEEE. find
  33. Mourad, A. and K. Darwish, “Subjectivity and sentiment analysis of modern standard Arabic and Arabic microblogs”. in Proceedings of the 4th workshop on computational approaches to subjectivity, sentiment and social media analysis. 2013. find
  34. El-Makky, N., et al., “Sentiment analysis of colloquial Arabic tweets”. in ASE BigData/SocialInformatics/PASSAT/BioMedCom 2014 Conference, Harvard University. 2014. find
  35. Omar, N., et al., “A comparative study of feature selection and machine learning algorithms for arabic sentiment classification”. in Asia information retrieval symposium. 2014. Springer. find
  36. Mierswa, I., et al., “Yale: Rapid prototyping for complex data mining tasks”. in Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining. 2006. ACM. find
  37. Pak, A. and P. Paroubek, “Twitter as a corpus for sentiment analysis and opinion mining”. in LREc. 2010. our card
  38. Shang, W., et al., “A novel feature selection algorithm for text categorization”. Expert Systems with Applications, 2007. 33(1): p. 1-5. find
  39. Haralampieva, V. and G. Brown, “Evaluation of Mutual information versus Gini index for stable feature selection”. 2016. find
  40. Bron, E.E., et al., “Feature Selection Based on the SVM Weight Vector for Classification of Dementia”. IEEE J. Biomed. Health Informat, 2015. 19(5): p. 1617-1626. find
  41. Pang, B. and L. Lee, “A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts”. in Proceedings of the 42nd annual meeting on Association for Computational Linguistics. 2004. Association for Computational Linguistics. our card
  42. Cortes, C. and V. Vapnik, “Support-vector networks”. Machine Leaming, 1995. 20(3): p. 273-297. find
  43. Joachims, T., “Text categorization with support vector machines: Learning with many relevant features”. in European conference on machine learning. 1998. Springer. our card
  44. Chang, C.-C. and C.-J. Lin. LIBSVM: a library for support vector machines. 2001 [cited 2 3]; 27]. Available from: www.csie.ntu.edu.tw/~cjlin/libsvm/faq.html. Omar Al-Harbi received the Ph.D. degree in Computer Science from Islamic Science University of Malaysia (USIM) in 2013. During 1997-1999, he is currently an assistant professor at the Department of Computer Science at Community College/Jazan University, Saudi Arabia Kingdom. His current research interests include word sense disambiguation, ontology, information retrieval, sentiment analysis, and question answering systems. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/al-harbi-2019-comparative-study-feature-selection-methods/