Elamur Research⌕
☰
home / papers / 2021 / Effect of Word Embedding Variable Parameters on Arabic Sentiment Analysis Performance

Effect of Word Embedding Variable Parameters on Arabic Sentiment Analysis Performance

Summary

The paper examines how the tunable settings of document embeddings affect sentiment classification of Arabic text. Arabic is hard for English-style pipelines because of rich morphology, dialects, short vowels and letters with several written forms. The authors pooled about 33K labelled reviews of hotels, restaurants, movies and products from the multi-domain resource of ElSahar and El-Beltagy (2015) and cleaned them with NLTK. Each review was then turned into a Doc2Vec vector, with the embedding model trained on the Abu El-Khair (2016) corpus. They varied three settings under both DBOW and DM-PV: context window, vector dimension and number of negative samples. The resulting vectors went to four standard classifiers, evaluated on an 80:20 split. Narrow windows worked best, while bigger vectors and more negative samples gave small gains. The best accuracy, 0.94, came from Logistic Regression on DBOW vectors. The work offers a documented starting grid for embedding hyperparameters in a morphologically rich language. The authors caution that the best settings shift with the background corpus, the class balance and the classifier.

Key points

  • Grid explored: window ∈ {1, 5, 10}, dimension ∈ {100, 300, 500}, negative samples ∈ {2, 5, 10, 30}, under both DBOW and DM-PV (PV-DM).
  • Classifiers: Logistic Regression, Decision Tree, SVM and Naive Bayes, all run under identical conditions and scored with Accuracy, Precision, Recall and F1.
  • Data: four domains (HTL, RES, MOV, PROD) totalling roughly 33K reviews, all strongly skewed toward positive labels (e.g., HTL has 10049 positive vs 2470 negative).
  • Implementations: Linan Qiu's modified version of the Le and Mikolov Doc2Vec code plus Gensim, both adapted to handle Arabic encoding.
  • Best accuracy was 0.93 (LR and SVM) with PV-DM and 0.94 (LR) with PV-DBOW.

Interesting ideas

  • I1. The same set of embedding settings is scored through several different classifiers, which shows whether a gain comes from the representation itself or only suits one model. — For Mova (learn): Our evaluation harness could sweep a feature or embedding setting across both the perceptron parser and gparse, so we keep only changes that help both witnesses.
  • I2. Before any representation is learned, the text is normalized: diacritics, noise characters and letter repetitions are removed so variant spellings collapse to one form. — For Mova (tokens): A similar normalization layer in crate en could fold elongations ("soooo"), quote and apostrophe variants, and hyphenation variants before tagging, with the original form kept for explainability.
  • I3. The embedding model is trained on a large unlabelled background corpus and then applied to a much smaller labelled task set, separating general lexical knowledge from the supervised task. — For Mova (tokens): Distributional vectors from a large English corpus could serve as a back-off signal for the tagger and lemmatizer on unseen words, without replacing the rule-based, explainable path.
  • I4. The authors explicitly warn that the best configuration depends on the background corpus, the polarity balance and the classifier, and they report per-domain class counts. — For Mova (learn): Every evaluation report could list label and class distributions next to the scores, alongside a majority-class negative control, so skewed data cannot pass for real skill.
  • I5. A narrow context window gave the best results here, suggesting that local context carried the useful signal for this task. — For Mova (tree): When gparse or the perceptron parser gets lexical-similarity features, the window size should be treated as a tuned hyperparameter, starting from small windows rather than wide topical ones.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2101.02906 · arXiv 2101.02906

@misc{alnawas2021effect,
  title = {{Effect of Word Embedding Variable Parameters on Arabic Sentiment Analysis Performance}},
  author = {Alnawas, Anwar and ARICI, Nursal},
  year = {2021},
  doi = {10.48550/arxiv.2101.02906},
  eprint = {2101.02906},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2101.02906}
}

References

14 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abu El-Khair I (2016) 1.5 billion words Arabic Corpus arXiv preprint arXiv:161104033 doi:2016arXiv161104033A find
  2. Al-Azani S, El-Alfy E-SM (2017) Using word embedding and ensemble learning for highly imbalanced data sentiment analysis in short arabic text. In: Proceedings of The 8th International Conference on Ambient Systems, Networks and Technologies, ANT Madeira, Portugal, 2017. Procedia Computer Science, pp 359-366. doi:10.1016/j.procs.2017.05.365 ↗
  3. Al-Sallab A, Baly R, Hajj H, Shaban KB, El-Hajj W, Badaro G (2017) Aroma: A recursive deep learning model for opinion mining in arabic as a low resource language ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP) 16:25 doi:10.1145/3086575 ↗
  4. Alnawas A, Arıcı N (2018) The Corpus Based Approach to Sentiment Analysis in Modern Standard Arabic and Arabic Dialects: A Literature Review Journal of Polytechnic 21:461-470 doi:10.2339/politeknik.403975 ↗
  5. Altowayan AA, Tao L (2016) Word embeddings for Arabic sentiment analysis. In: Proceedings of IEEE International Conference on Big Data, Washington, DC, USA, 2016. IEEE, pp 3820-3825. doi:10.1109/BigData.2016.7841054 ↗
  6. Bansal M, Gimpel K, Livescu K (2014 ) Tailoring continuous word representations for dependency parsing. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Baltimore, Maryland, USA, 2014. pp 809-815. doi:10.3115/v1/P14-2131 ↗ our card
  7. Dahou A, Xiong S, Zhou J, Haddoud MH, Duan P (2016) Word embeddings and convolutional neural network for arabic sentiment classification. In: Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers, Osaka,Japan, 2016. pp 2418-2427 Duwairi R, Ahmed NA, Al-Rifai SY (2015) Detecting sentiment embedded in Arabic social media–a lexicon-based approach Journal of Intelligent & Fuzzy Systems 29:107-117 doi:10.3233/IFS-151574 ↗
  8. ElSahar H, El-Beltagy SR (2015) Building large arabic multi-domain resources for sentiment analysis. In: Proceedings of 16th International Conference on Intelligent Text Processing and Computational Linguistics, Cairo, Egypt, 2015. Springer, pp 23-34. doi:10.1007/978-3-319-18117-2_2 ↗
  9. Godin F, Vandersmissen B, De Neve W, Van de Walle R (2015) Multimedia Lab @ ACL WNUT NER Shared Task: Named Entity Recognition for Twitter Microposts using Distributed Word Representations. In: Proceedings of the Workshop on Noisy User-generated Text, Beijing, China, 2015. pp 146-153. doi:10.18653/v1/W15-4322 ↗
  10. Le Q, Mikolov T (2014) Distributed representations of sentences and documents. In: Proceedings of International Conference on Machine Learning, Beijing, China, 2014. pp 1188-1196. doi:2014arXiv1405.4053L ↗ our card
  11. Mikolov T, Chen K, Corrado G, Dean J (2013a) Efficient estimation of word representations in vector space arXiv preprint arXiv:13013781 doi:2013arXiv1301.3781M ↗
  12. Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Proceedings of Advances in Neural Information Processing Systems 26, Nivada.USA, 2013b. pp 3111-3119. doi:2013arXiv1310.4546M ↗
  13. Qiu L (2015) Sentiment Analysis using Doc2Vec in gensim. https://github.com.https://github.com/linanqiu/word2vec-sentiments. Accessed December 2017 Tang D, Wei F, Yang N, Zhou M, Liu T, Qin B (2014) Learning sentiment-specific word embedding for twitter sentiment classification. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, Maryland, USA, 2014. pp 1555-1565. doi:10.3115/v1/P14-1146 ↗ our card
  14. Zahran MA, Magooda A, Mahgoub AY, Raafat H, Rashwan M, Atyia A (2015) Word representations in vector space and their applications for arabic. In: Proceedings of the International Conference on Intelligent Text Processing and Computational Linguistics, Cairo, Egypt, 2015. Springer, pp 430-443. doi:10.1007/978-3-319-18111-0_32 ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/alnawas-2021-effect-word-embedding-variable-parameters/