Elamur Research⌕
☰
home / papers / 2018 / Multi-grained Attention Network for Aspect-Level Sentiment Classification

Multi-grained Attention Network for Aspect-Level Sentiment Classification

Paper · 2018 · Feifan Fan, Yansong Feng and Dongyan Zhao

Summary

The paper addresses aspect-level sentiment classification, which means predicting whether a sentence expresses positive, neutral or negative sentiment toward a specific aspect term. Earlier attention models build their attention guides by averaging the aspect or context vectors. The authors argue that this "coarse" pooling discards information when aspects span several words or contexts are long. They propose MGAN, which runs a BiLSTM over GloVe embeddings and adds position weighting so that words near the aspect count more. MGAN combines two kinds of attention. Coarse-grained bidirectional attention (C-Aspect2Context, C-Context2Aspect) works from pooled vectors. Fine-grained attention (F-Aspect2Context, F-Context2Aspect) uses a word-by-word alignment matrix. MGAN also adds an aspect alignment loss, which pushes apart the attention distributions of aspects that share a sentence but differ in polarity. The authors present this as the first use of such inter-aspect interactions. On SemEval 2014 laptop and restaurant reviews and a Twitter set, MGAN beats strong baselines such as RAM, MemNet and IAN. This shows that word-level interactions and joint modelling of co-occurring aspects carry useful signal.

Key points

  • Main results: MGAN reports the following accuracy / Macro-F1 scores:
    • Laptop: 0.7539 / 0.7247 (best baseline RAM: 0.7449 / 0.7135).
    • Restaurant: 0.8125 / 0.7194.
    • Twitter: 0.7254 / 0.7081 (best baseline BILSTM-ATT-G: 0.7038 accuracy).
  • Ablation on laptop: accuracy rises from MGAN-C (0.7273) to MGAN-F (0.7398), then MGAN-CF (0.7445), then full MGAN (0.7539). The fine-grained attention helps most on laptop, which has the largest share of aspects longer than two words (9.24%).
  • Alignment loss and Twitter: the alignment loss adds nothing on Twitter. Nearly all tweets (99.91%) contain a single aspect, so MGAN there is equivalent to MGAN-CF.
  • Multi-aspect sentences: these are common in the review data. About 36% of laptop sentences and 49% of restaurant sentences have two or more aspects, which is where the alignment loss pays off.
  • Case study: attention visualisation on a sentence about "resolution" and "fonts" shows that the alignment loss widens the attention gaps on polarity-bearing words such as "small" and "higher".

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Peking UniversityBeijing · China100 works · 35,618 citations🔬Beijing Institute of Big Data ResearchBeijing · China2 works · 839 citations

📚 Shelves 4 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works

🧭 Topics 1 Sentiment & emotion · 608 works

Cite

Cite as Feifan Fan, Yansong Feng and Dongyan Zhao. 2018. Multi-grained Attention Network for Aspect-Level Sentiment Classification. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d18-1380

@inproceedings{fan2018multi,
  title = {{Multi-grained Attention Network for Aspect-Level Sentiment Classification}},
  author = {Feifan Fan, Yansong Feng and Dongyan Zhao},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1380},
  url = {https://doi.org/10.18653/v1/d18-1380}
}

References

22 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Peng Chen, Zhongqian Sun, Lidong Bing, and Wei Yang. 2017. Recurrent attention network on memory for aspect sentiment analysis. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 452–461. our card
  2. Li Dong, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, and Ke Xu. 2014. Adaptive recursive neural network for target-dependent twitter sentiment classification. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 49–54. our card
  3. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. find
  4. Long Jiang, Mo Yu, Ming Zhou, Xiaohua Liu, and Tiejun Zhao. 2011. Target-dependent twitter sentiment classification. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 151–160. Association for Computational Linguist… find
  5. Svetlana Kiritchenko, Xiaodan Zhu, Colin Cherry, and Saif Mohammad. 2014. Nrc-canada-2014: Detecting aspects and sentiment in customer reviews. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 437– 442. our card
  6. Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2016. Rationalizing neural predictions. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 107–117. our card
  7. Cheng Li, Xiaoxiao Guo, and Qiaozhu Mei. 2017. Deep memory networks for attitude identification. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, pages 671– 680. ACM. find
  8. Bing Liu. 2012. Sentiment analysis and opinion mining. Synthesis lectures on human language technologies, 5(1):1–167. find
  9. Jiangming Liu and Yue Zhang. 2017. Attention modeling for targeted sentiment. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, volume 2, pages 572–577. find
  10. Dehong Ma, Sujian Li, Xiaodong Zhang, and Houfeng Wang. 2017. Interactive attention networks for aspect-level sentiment classification. In IJCAI. find
  11. Thien Hai Nguyen and Kiyoaki Shirai. 2015. Phrasernn: Phrase recursive neural network for aspect-based sentiment analysis. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2509–2514. find
  12. Boyuan Pan, Hao Li, Zhou Zhao, Bin Cao, Deng Cai, and Xiaofei He. 2017. Memen: Multi-layer embedding with memory networks for machine comprehension. arXiv preprint arXiv:1707.09098. — [arXiv:1707.09098](https://arxiv.org/abs/1707.09098) find
  13. Bo Pang, Lillian Lee, et al. 2008. Opinion mining and sentiment analysis. Foundations and Trends R in Information Retrieval, 2(1–2):1–135. find
  14. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543. our card
  15. Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. Semeval-2014 task 4: Aspect based sentiment analysis. Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 27–35. our card
  16. Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. Bidirectional attention flow for machine comprehension. In Proceedings of ICLR. our card
  17. Richard Socher, Jeffrey Pennington, Eric H Huang, Andrew Y Ng, and Christopher D Manning. 2011. Semi-supervised recursive autoencoders for predicting sentiment distributions. In Proceedings of the conference on empirical methods in natural language processing, pages 151–161. Association for Computat… our card
  18. Duyu Tang, Bing Qin, Xiaocheng Feng, and Ting Liu. 2016a. Effective lstms for target-dependent sentiment classification. pages 3298–3307. our card
  19. Duyu Tang, Bing Qin, and Ting Liu. 2016b. Aspect level sentiment classification with deep memory network. pages 214–224. our card
  20. Duy-Tin Vo and Yue Zhang. 2015. Target-dependent twitter sentiment classification with rich automatic features. In IJCAI, pages 1347–1353. find
  21. Joachim Wagner, Piyush Arora, Santiago Cortes, Utsab Barman, Dasha Bogdanova, Jennifer Foster, and Lamia Tounsi. 2014. Dcu: Aspect-based polarity classification for semeval task 4. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 223–229. find
  22. Yequan Wang, Minlie Huang, Li Zhao, et al. 2016. Attention-based lstm for aspect-level sentiment classification. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 606–615. our card

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/fan-2018-multi-grained-attention-network-aspect/