Elamur Research⌕
☰
home / papers / 2021 / Arabic aspect based sentiment analysis using bidirectional GRU based models

Arabic aspect based sentiment analysis using bidirectional GRU based models

Summary

The paper tackles aspect-based sentiment analysis (ABSA) for Arabic. Most earlier Arabic work used classical machine learning that depends on scarce resources such as sentiment lexicons. The authors drop these resources and handcrafted features and use two deep models built on Gated Recurrent Units. The first, BGRU-CNN-CRF, handles opinion target extraction (T2). It combines character features from a CNN with word representations, feeds them to a bidirectional GRU, and adds a CRF layer on top to label aspect spans. The second, IAN-BGRU, handles aspect polarity classification (T3). It is an interactive attention network where the target and its context attend to each other, with bidirectional GRUs replacing the original unidirectional LSTM. Both models are tested on the SemEval-2016 Arabic hotel reviews benchmark (Pontiki et al., 2016). They reach 70.67% F1 on T2 and 83.98% accuracy on T3, beating the SVM baseline and several neural rivals. The results show that competitive fine-grained Arabic sentiment analysis is possible without lexicons.

Key points

  • Data: the Arabic hotel reviews dataset from SemEval-2016 ABSA (Pontiki et al., 2016). The tasks are T2 (opinion target extraction, OTE) and T3 (aspect polarity). T1 (aspect category) is not addressed.

  • T2: BGRU-CNN-CRF reaches 70.67% F1, reported as a 39.7% improvement over the baseline.

  • T3: IAN-BGRU reaches 83.98% accuracy, against 76.4 for the baseline (+7.58).

  • T3 scores for the comparison systems:

    SystemAccuracy
    INSIGHT-1 (CNN)82.71
    LSTM81.49
    TD-LSTM81.79
    AB-LSTM-PC82.60
    MEMNET82.64
    IAN-LSTM83.18
    IAN-GRU83.68
    IAN-BLSTM83.48

  • The multi-hop attention of MEMNET brought no gain over the IAN variants. The authors attribute this to model complexity and to the lack of a mechanism for stopping the attention steps.

Interesting ideas

  • I1. The target phrase and the surrounding sentence each build an attention summary guided by the other, so the representation of one shapes how the other is read; this suits any decision that concerns one item inside a larger sentence. — For Mova (frames): when assigning an evaluative stance or semantic role to a specific argument, score the argument span and the predicate context against each other rather than reading the whole clause uniformly.
  • I2. Character-level CNN features are fused with word vectors under a CRF, so that unseen word forms still get useful features and the predicted spans stay well-formed. — For Mova (tokens): a character-derived feature channel could help the tagger with rare or misspelled forms, while a structured decoder keeps multi-token labels consistent.
  • I3. The authors run a controlled ladder of architecture swaps under one fixed frame: LSTM vs GRU and unidirectional vs bidirectional. This isolates what each change contributes, and it showed that the simpler GRU trained better than LSTM. — For Mova (learn): evaluate parser or tagger changes as single-factor swaps on the same data, so each gain can be attributed and explained.
  • I4. Qualitative checks use one sentence with two aspects of opposite polarity, and another where "not" flips the meaning, as probes the model must pass. — For Mova (learn): build small minimal-contrast sets (same sentence, negated or with aspects swapped) as negative controls, so a reader cannot score well by ignoring structure.
  • I5. The pipeline first extracts the opinion targets and then judges sentiment for each extracted target separately. This keeps two opinions about different things in one sentence from blending together. — For Mova (world): record evaluations as properties attached to individual entities in the text world, so mixed reviews yield per-entity states instead of one sentence-level verdict.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

3 papers of our library cite it (of the 140884 whose bibliographies we read); 1 of them have a page here, 2 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1016/j.jksuci.2021.08.030 · arXiv 2101.10539

@misc{mabdelgwad2021arabic,
  title = {{Arabic aspect based sentiment analysis using bidirectional GRU based models}},
  author = {M.Abdelgwad, Mohammed and Soliman, Taysir Hassan A and I.Taloba, Ahmed and Farghaly, Mohamed Fawzy},
  year = {2021},
  doi = {10.1016/j.jksuci.2021.08.030},
  eprint = {2101.10539},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1016/j.jksuci.2021.08.030}
}

References

62 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Al-Dabet, S., Tedmori, S., Al-Smadi, M., 2020. Extracting opinion targets using attention-based neural model. SN Computer Science. 1(5), 1–10. find
  2. Al-Smadi, M., Al-Ayyoub, M., Jararweh, Y., Qawasmeh, O., 2019a. Enhancing aspect-based sentiment analysis of arabic hotels’ reviews using morphological, syntactic and semantic features. Information Processing & Management. 56(2), 308–319. find
  3. Al-Smadi, M., Qawasmeh, O., Al-Ayyoub, M., Jararweh, Y., Gupta, B., 2018. Deep recurrent neural network vs. support vector machine for aspect-based sentiment analysis of arabic hotels’ reviews. Journal of computational science. 27, 386–393. find
  4. Al-Smadi, M., Qawasmeh, O., Talafha, B., Quwaider, M., 2015. Human annotated arabic dataset of book reviews for aspect based sentiment analysis. In: 2015 3rd International Conference on Future Internet of Things and Cloud. IEEE. pp. 726–730. find
  5. Al-Smadi, M., Talafha, B., Al-Ayyoub, M., Jararweh, Y., 2019b. Using long short-term memory deep neural networks for aspect-based sentiment analysis of arabic reviews. International Journal of Machine Learning and Cybernetics. 10(8), 2163–2175. find
  6. Algihab, W., Alawwad, N., Aldawish, A., AlHumoud, S., 2019. Arabic speech recognition with deep learning: A review. In: International Conference on Human-Computer Interaction. Springer. pp. 15–31. find
  7. Ameur, M. S. H., Meziane, F., Guessoum, A., 2017. Arabic machine transliteration using an attention-based encoder-decoder model. Procedia Computer Science. 117, 287–297. find
  8. Bengio, Y., Simard, P., Frasconi, P., 1994. Learning long-term dependencies with gradient descent is difficult. IEEE transactions on neural networks. 5(2), 157–166. find
  9. Bojanowski, P., Grave, E., Joulin, A., Mikolov, T., 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics. 5, 135–146. find
  10. Caruana, R., Lawrence, S., Giles, C. L., 2001. Overfitting in neural nets: Backpropagation, conjugate gradient, and early stopping. In: Advances in neural information processing systems. pp. 402–408. find
  11. Chen, P., Sun, Z., Bing, L., Yang, W., 2017a. Recurrent attention network on memory for aspect sentiment analysis. In: Proceedings of the 2017 conference on empirical methods in natural language processing. pp. 452–461. find our card
  12. Chen, T., Xu, R., He, Y., Wang, X., 2017b. Improving sentiment analysis via sentence type classification using bilstm-crf and cnn. Expert Systems with Applications. 72, 221–230. find
  13. Chiu, J. P. Nichols, E., 2016. Named entity recognition with bidirectional lstm-cnns. Transactions of the Association for Computational Linguistics. 4, 357–370. find our card
  14. Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., Bengio, Y., 2014. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078. ↗
  15. Da’u, A. Salim, N., 2019. Aspect extraction on user textual reviews using multi-channel convolutional neural network. PeerJ Computer Science. 5, e191. find
  16. Devlin, J., Chang, M.-W., Lee, K., Toutanova, K., 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. ↗
  17. Ding, X. Liu, B., 2007. The utility of linguistic rules in opinion mining. In: Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval. pp. 811–812. find
  18. Dong, L., Wei, F., Tan, C., Tang, D., Zhou, M., Xu, K., 2014. Adaptive recursive neural network for target-dependent twitter sentiment classification. In: Proceedings of the 52nd annual meeting of the association for computational linguistics (volume 2: Short papers). pp. 49–54. find our card
  19. Goodfellow, I., Bengio, Y., Courville, A., 2016. Deep learning. MIT press. find
  20. Hochreiter, S. Schmidhuber, J., 1997. Long short-term memory. Neural computation. 9(8), 1735–1780. find
  21. Hu, M. Liu, B., 2004. Mining and summarizing customer reviews. In: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining. pp. 168–177. find
  22. Huang, B., Ou, Y., Carley, K. M., 2018. Aspect level sentiment classification with attention-over-attention neural networks. In: International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation. Springer. pp. 197–206. find
  23. Irsoy, O. Cardie, C., 2014. Opinion mining with deep recurrent neural networks. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). pp. 720–728. find
  24. Khalifa, M. Shaalan, K., 2019. Character convolutions for arabic named entity recognition with long short-term memory networks. Computer Speech & Language. 58, 335–346. find
  25. Kingma, D. P. Ba, J., 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. ↗
  26. Kwaik, K. A., Saad, M., Chatzikyriakidis, S., Dobnik, S., 2019. Lstm-cnn deep learning model for sentiment analysis of dialectal arabic. In: International Conference on Arabic Language Processing. Springer. pp. 108–121. find
  27. Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., Dyer, C., 2016. Neural architectures for named entity recognition. arXiv preprint arXiv:1603.01360. ↗
  28. LeCun, Y., Bengio, Y., Hinton, G., 2015. Deep learning. nature. 521(7553), 436–444. find
  29. Li, H., Min, M. R., Ge, Y., Kadav, A., 2017. A context-aware attention network for interactive question answering. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. pp. 927–935. find
  30. Li, Q., Zhang, X., Xiong, J., Hwu, W.-m., Chen, D., 2019a. Implementing neural machine translation with bi-directional gru and attention mechanism on fpgas using hls. In: Proceedings of the 24th Asia and South Pacific Design Automation Conference. pp. 693–698. find
  31. Li, X., Bing, L., Lam, W., Shi, B., 2018. Transformation networks for target-oriented sentiment classification. arXiv preprint arXiv:1805.01086. ↗ our card
  32. Li, X., Bing, L., Zhang, W., Lam, W., 2019b. Exploiting bert for end-to-end aspect-based sentiment analysis. arXiv preprint arXiv:1910.00883. ↗
  33. Liu, P., Joty, S., Meng, H., 2015. Fine-grained opinion mining with recurrent neural networks and word embeddings. In: Proceedings of the 2015 conference on empirical methods in natural language processing. pp. 1433–1443. find our card
  34. Liu, Q., Zhang, H., Zeng, Y., Huang, Z., Wu, Z., 2018. Content attention model for aspect based sentiment analysis. In: Proceedings of the 2018 World Wide Web Conference. pp. 1023–1032. find
  35. Luo, L.-x., 2019. Network text sentiment analysis method combining lda text representation and gru-cnn. Personal and Ubiquitous Computing. 23(3-4), 405–412. find
  36. Ma, D., Li, S., Zhang, X., Wang, H., 2017. Interactive attention networks for aspect-level sentiment classification. arXiv preprint arXiv:1709.00893. ↗
  37. Ma, X. Hovy, E., 2016. End-to-end sequence labeling via bi-directional lstm-cnns-crf. arXiv preprint arXiv:1603.01354. ↗ our card
  38. Meng, F., Lu, Z., Li, H., Liu, Q., 2016. Interactive attention for neural machine translation. arXiv preprint arXiv:1610.05011. ↗
  39. Mikolov, T., Chen, K., Corrado, G., Dean, J., 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. ↗
  40. Nasukawa, T. Yi, J., 2003. Sentiment analysis: Capturing favorability using natural language processing. In: Proceedings of the 2nd international conference on Knowledge capture. pp. 70–77. find
  41. Nguyen, T. H. Shirai, K., 2015. Phrasernn: Phrase recursive neural network for aspect-based sentiment analysis. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. pp. 2509–2514. find
  42. Pang, B., Lee, L., Vaithyanathan, S., 2002. Thumbs up? sentiment classification using machine learning techniques. arXiv preprint cs/0205070. find our card
  43. Pascanu, R., Mikolov, T., Bengio, Y., 2013. On the difficulty of training recurrent neural networks. In: International conference on machine learning. pp. 1310–1318. find
  44. Pontiki, M., Galanis, D., Papageorgiou, H., Androutsopoulos, I., Manandhar, S., Al-Smadi, M., Al-Ayyoub, M., Zhao, Y., Qin, B., De Clercq, O., et al., 2016. Semeval-2016 task 5: Aspect based sentiment analysis. In: International workshop on semantic evaluation. pp. 19–30. find our card
  45. Pontiki, M., Galanis, D., Papageorgiou, H., Manandhar, S., Androutsopoulos, I., 2015. Semeval-2015 task 12: Aspect based sentiment analysis. In: Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015). pp. 486–495. find our card
  46. Pontiki, M., Galanis, D., Pavlopoulos, J., Papageorgiou, H., Androutsopoulos, I., Manandhar, S., 2014. SemEval-2014 task 4: Aspect based sentiment analysis. In: Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). Association for Computational Linguistics. Dublin, Ireland. pp. 27–35. find our card
  47. Poria, S., Cambria, E., Gelbukh, A., 2016. Aspect extraction for opinion mining with a deep convolutional neural network. Knowledge-Based Systems. 108, 42–49. find
  48. Rana, T. A. Cheah, Y.-N., 2016. Aspect extraction in sentiment analysis: comparative analysis and survey. Artificial Intelligence Review. 46(4), 459–483. find
  49. Ruder, S., Ghaffari, P., Breslin, J. G., 2016a. A hierarchical model of reviews for aspect-based sentiment analysis. arXiv preprint arXiv:1609.02745. ↗
  50. Ruder, S., Ghaffari, P., Breslin, J. G., 2016b. Insight-1 at semeval-2016 task 5: Deep learning for multilingual aspect-based sentiment analysis. arXiv preprint arXiv:1609.02748. ↗
  51. Soliman, A. B., Eissa, K., El-Beltagy, S. R., 2017. Aravec: A set of arabic word embedding models for use in arabic nlp. Procedia Computer Science. 117, 256–265. find
  52. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R., 2014. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research. 15(1), 1929–1958. find
  53. Tang, D., Qin, B., Feng, X., Liu, T., 2015. Effective lstms for target-dependent sentiment classification. arXiv preprint arXiv:1512.01100. ↗ our card
  54. Tang, D., Qin, B., Liu, T., 2016. Aspect level sentiment classification with deep memory network. arXiv preprint arXiv:1605.08900. ↗ our card
  55. Wang, B., Liu, K., Zhao, J., 2016a. Inner attention based recurrent neural networks for answer selection. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 1288–1297. find
  56. Wang, Y., Huang, M., Zhu, X., Zhao, L., 2016b. Attention-based lstm for aspect-level sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing. pp. 606–615. find our card
  57. Xue, W. Li, T., 2018. Aspect based sentiment analysis with gated convolutional networks. arXiv preprint arXiv:1805.07043. ↗
  58. Yang, M., Qu, Q., Chen, X., Guo, C., Shen, Y., Lei, K., 2018. Feature-enhanced attention network for target-dependent sentiment classification. Neurocomputing. 307, 91–97. find
  59. Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E., 2016. Hierarchical attention networks for document classification. In: Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies. pp. 1480–1489. find our card
  60. Zerari, N., Abdelhamid, S., Bouzgou, H., Raymond, C., 2019. Bidirectional deep architecture for arabic speech recognition. Open Computer Science. 9(1), 92–102. find
  61. Zhai, Z., Nguyen, D. Q., Verspoor, K., 2018. Comparing cnn and lstm character-level embeddings in bilstm-crf models for chemical and disease named entity recognition. arXiv preprint arXiv:1808.08450. ↗
  62. Zhao, J., Liu, K., Xu, L., 2016. Sentiment analysis: mining opinions, sentiments, and emotions. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/m-2021-abdelgwad-arabic-aspect-based-sentiment-analysis/