Elamur Research
☰
home / papers / 2019 / Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence

Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/N19-1035.pdf

Summary

Aspect-based sentiment analysis (ABSA) assigns a polarity to each aspect mentioned in a text. Its targeted variant (TABSA) also links aspects to specific entities, which matters when one review discusses several targets. The authors found that fine-tuning BERT directly on single sentences gives little gain on these tasks. They instead recast the problem as sentence-pair classification, similar to QA or NLI. For each target–aspect combination they generate a second input sentence in one of four styles: a natural question or a terse pseudo-sentence, each with or without the candidate polarity label. Label-bearing versions become yes/no decisions per polarity. They fine-tune uncased BERT-base on these pairs and test on SentiHood (TABSA) and SemEval-2014 Task 4 (aspect category detection and polarity). All four pair-based variants clearly beat earlier feature-based, LSTM and memory-network systems, as well as a single-sentence BERT baseline. This shows the gain comes from the reformulation, not just from BERT. The idea is simple and transfers readily to other aspect-level classification problems. Code is released on GitHub (HSLCY/ABSA-BERT-pair).

Key points

  • Four auxiliary-sentence models: BERT-pair-QA-M, BERT-pair-NLI-M, BERT-pair-QA-B, BERT-pair-NLI-B. The "-B" variants add the polarity label and choose the class with the highest "yes" probability.
  • SentiHood has 5,215 sentences, of which 3,862 have a single target. Evaluation covers four aspects: general, price, transit-location and safety.
  • On SentiHood, the best pair models reach 87.9 aspect Macro-F1 (QA-B) and 93.6 sentiment accuracy (QA-M). This is +9.4 F1 and +2.6 accuracy over Dmu-Entnet. BERT-single gets only 85.5 sentiment accuracy.
  • On SemEval-2014 Task 4, BERT-pair-NLI-B reaches 92.18 F1 on Subtask 3, compared with 88.58 for NRC-Canada. BERT-pair-QA-B gets 85.9 / 89.9 / 95.6 accuracy on Subtask 4 in the 4-way / 3-way / binary settings.
  • Setup: uncased BERT-base (12 layers, hidden size 768, 12 heads, 110M parameters), fine-tuned for 4 epochs with learning rate 2e-5, batch size 24 and dropout 0.1.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

121 papers of our library cite it (of the 140673 whose bibliographies we read); 4 of them have a page here, 117 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n19-1035

@inproceedings{sun2019utilizing,
  title = {{Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence}},
  author = {Sun, Chi and Huang, Luyao and Qiu, Xipeng},
  booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)},
  year = {2019},
  doi = {10.18653/v1/n19-1035},
  url = {https://doi.org/10.18653/v1/n19-1035}
}

References

20 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Suresh Manandhar, and Ion Androutsopoulos. 2015. Semeval-2015 task 12: Aspect based sentiment analysis. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 486–495. find
  2. Caroline Brun, Diana Nicoleta Popa, and Claude Roux. 2014. Xrce: Hybrid classification for aspect-based sentiment analysis. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 838–842. find
  3. Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. Semeval-2014 task 4: Aspect based sentiment analysis. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 27–35. Association for Computat… find
  4. Erik Cambria, Soujanya Poria, Rajiv Bajpai, and Björn Schuller. 2016. Senticnet 4: A semantic resource for sentiment analysis based on conceptual primitives. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2666–2677. find
  5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. find
  6. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. URL https://s3-us-west-2. amazonaws. com/openai-assets/research-covers/language-unsupervised/language understanding paper. pdf. find
  7. Mikael Henaff, Jason Weston, Arthur Szlam, Antoine Bordes, and Yann LeCun. 2016. Tracking the world state with recurrent entity networks. arXiv preprint arXiv:1612.03969. find
  8. Marzieh Saeidi, Guillaume Bouchard, Maria Liakata, and Sebastian Riedel. 2016. Sentihood: targeted aspect based sentiment analysis dataset for urban neighbourhoods. arXiv preprint arXiv:1610.03771. find
  9. Yohan Jo and Alice H Oh. 2011. Aspect and sentiment unification model for online review analysis. In Proceedings of the fourth ACM international conference on Web search and data mining, pages 815– 824. ACM. find
  10. Duyu Tang, Bing Qin, Xiaocheng Feng, and Ting Liu. 2015. Effective lstms for target-dependent sentiment classification. arXiv preprint arXiv:1512.01100. find
  11. Svetlana Kiritchenko, Xiaodan Zhu, Colin Cherry, and Saif Mohammad. 2014. Nrc-canada-2014: Detecting aspects and sentiment in customer reviews. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 437– 442. find
  12. Duyu Tang, Bing Qin, and Ting Liu. 2016. Aspect level sentiment classification with deep memory network. arXiv preprint arXiv:1605.08900. find
  13. Joachim Wagner, Piyush Arora, Santiago Cortes, Utsab Barman, Dasha Bogdanova, Jennifer Foster, and Lamia Tounsi. 2014. Dcu: Aspect-based polarity classification for semeval task 4. In Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), pages 223–229. find
  14. Fei Liu, Trevor Cohn, and Timothy Baldwin. 2018. Recurrent entity networks with delayed memory update for targeted aspect-based sentiment analysis. arXiv preprint arXiv:1804.11019. find
  15. Yukun Ma, Haiyun Peng, and Erik Cambria. 2018. Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive lstm. In Proceedings of AAAI. find
  16. Bo Wang, Maria Liakata, Arkaitz Zubiaga, and Rob Procter. 2017. Tdparse: Multi-target-specific sentiment recognition on twitter. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, volume 1, pages 483–493. find
  17. Thien Hai Nguyen and Kiyoaki Shirai. 2015. Phrasernn: Phrase recursive neural network for aspect-based sentiment analysis. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2509–2514. find
  18. Yequan Wang, Minlie Huang, Li Zhao, et al. 2016. Attention-based lstm for aspect-level sentiment classification. In Proceedings of the 2016 conference on empirical methods in natural language processing, pages 606–615. find
  19. Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv preprint arXiv:1802.05365. find
  20. Maria Pontiki, Dimitris Galanis, Haris Papageorgiou, Ion Androutsopoulos, Suresh Manandhar, AL-Smadi Mohammad, Mahmoud Al-Ayyoub, Yanyan Zhao, Bing Qin, Orphée De Clercq, et al. 2016. Semeval-2016 task 5: Aspect based sentiment analysis. In Proceedings of the 10th international workshop on semantic… find