Elamur Research⌕
☰
home / papers / 2019 / A Challenge Dataset and Effective Models for Aspect-Based Sentiment Analysis

A Challenge Dataset and Effective Models for Aspect-Based Sentiment Analysis

Summary

Standard aspect-based sentiment analysis (ABSA) benchmarks have a hidden weakness. In most of their sentences, either one aspect appears or all aspects share the same polarity. A model can therefore score well by predicting overall sentence sentiment and ignoring the target aspect. To fix this, the authors built MAMS (Multi-Aspect Multi-Sentiment) from Citysearch New York restaurant reviews. Every sentence in it mentions at least two aspects carrying different polarities. MAMS has versions for both aspect-term (ATSA) and aspect-category (ACSA) sentiment analysis. They also propose CapsNet, a capsule-network classifier with two additions: an aspect-aware normalization step, and a routing scheme guided by sentiment capsules initialized from sentiment-word embeddings. CapsNet-BERT is a variant that uses pre-trained BERT as its encoder. On MAMS, aspect-agnostic classifiers fall to roughly 50% accuracy, and several attention-based ABSA systems lose much of their advantage. CapsNet-BERT achieves the best scores across all tested datasets. MAMS thus serves as a stricter test of whether models truly link sentiment to specific aspects.

Key points

  • MAMS contains 13,854 ATSA instances and 8,879 ACSA instances. That is about 2.87× and 1.87× the size of SemEval-2014 Restaurant. Sentences average 2.62 aspect terms and 2.25 aspect categories, and the ACSA version uses eight categories.
  • In existing datasets, few sentences mix aspects with different polarities: 26.58% (Restaurant-ATSA), 9.58% (Restaurant-ACSA), 20.05% (Laptop) and 0.09% (Twitter). A MAMS-small split matches the Restaurant training size for fair comparison.
  • TextCNN reaches about 76% on Restaurant ATSA but only about 53% on MAMS ATSA. MemNet and AEN, which do not model word order, also perform poorly.
  • CapsNet-BERT outperforms vanilla BERT on MAMS: 83.39 vs. 82.22 (ATSA) and 79.46 vs. 78.29 (ACSA).
  • Replacing capsule-guided routing with standard dynamic routing (the -DR variants) lowers results on every dataset. Data and code are available at github.com/siat-nlp/MAMS-for-ABSA.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

87 papers of our library cite it (of the 140884 whose bibliographies we read); 2 of them have a page here, 85 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d19-1654

@inproceedings{jiang2019challenge,
  title = {{A Challenge Dataset and Effective Models for Aspect-Based Sentiment Analysis}},
  author = {Jiang, Qingnan and Chen, Lei and Xu, Ruifeng and Ao, Xiang and Yang, Min},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)},
  year = {2019},
  doi = {10.18653/v1/d19-1654},
  url = {https://doi.org/10.18653/v1/d19-1654}
}

References

20 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Langua… find
  2. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. find
  3. Li Dong, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, and Ke Xu. 2014. Adaptive recursive neural network for target-dependent twitter sentiment classification. In Proceedings of the 52nd annual meeting of the association for computational linguistics (volume 2: Short papers), volume 2, pages 49–54. find our card
  4. Gayatree Ganu, Nomie Elhadad, and Amlie Marian. 2009. Beyond the stars: Improving rating predictions using review text content. In WebDB. find
  5. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770– 778. find
  6. Binxuan Huang, Yanglan Ou, and Kathleen M Carley. 2018. Aspect level sentiment classification with attention-over-attention neural networks. In International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation, pages 197–… find
  7. Diederik P. Kingma and Jimmy Lei Ba. 2015. Adam: A method for stochastic optimization. international conference on learning representations. find
  8. Zeyang Lei, Yujiu Yang, Min Yang, Wei Zhao, Jun Guo, and Yi Liu. 2019. A human-like semantic cognition network for aspect-level sentiment classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6650–6657. find
  9. Suresh Manandhar. 2014. Semeval-2014 task 4: Aspect based sentiment analysis. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 27–35. find our card
  10. Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton. 2017. Dynamic routing between capsules. neural information processing systems, pages 3856–3866. find
  11. Youwei Song, Jiahai Wang, Tao Jiang, Zhiyue Liu, and Yanghui Rao. 2019. Attentional encoder network for targeted sentiment classification. arXiv preprint arXiv:1902.09314. — [arXiv:1902.09314](https://arxiv.org/abs/1902.09314) find
  12. Duyu Tang, Bing Qin, Xiaocheng Feng, and Ting Liu. 2016a. Effective lstms for target-dependent sentiment classification. international conference on computational linguistics, pages 3298–3307. find our card
  13. Duyu Tang, Bing Qin, and Ting Liu. 2016b. Aspect level sentiment classification with deep memory network. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 214–224. find our card
  14. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. neural information processing systems, pages 5998–6008. find
  15. Yequan Wang, Minlie Huang, xiaoyan zhu, and Li Zhao. 2016. Attention-based lstm for aspect-level sentiment classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 606–615. find our card
  16. Wei Xue and Tao Li. 2018. Aspect based sentiment analysis with gated convolutional networks. meeting of the association for computational linguistics, 1:2514–2523. find
  17. Min Yang, Qingnan Jiang, Ying Shen, Qingyao Wu, Zhou Zhao, and Wei Zhou. 2019. Hierarchical human-like strategy for aspect-level sentiment classification with sentiment linguistic knowledge and reinforcement learning. Neural Networks. find
  18. Dehong Ma, Sujian Li, Xiaodong Zhang, and Houfeng Wang. 2017. Interactive attention networks for aspect-level sentiment classification. In IJCAI’17 Proceedings of the 26th International Joint Conference on Artificial Intelligence, pages 4068–4074. find
  19. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543. find
  20. Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/jiang-2019-challenge-dataset-effective-models-aspect/