Elamur Research⌕
☰
home / papers / 2018 / Transformation Networks for Target-Oriented Sentiment Classification

Transformation Networks for Target-Oriented Sentiment Classification

Summary

The paper addresses target-oriented (aspect-level) sentiment classification: deciding whether a sentence expresses positive, negative or neutral sentiment toward a specific opinion target within it. The authors argue that attention-based RNNs, the dominant approach, can pull in irrelevant words as noise. Plain CNNs, meanwhile, cannot properly use target information or separate opinions about different targets in the same sentence. They propose TNet (Target-Specific Transformation Networks). It encodes the sentence with a BiLSTM and refines word vectors through stacked Context-Preserving Transformation (CPT) layers. Inside each layer, a Target-Specific Transformation (TST) builds a separate target representation for every context word and fuses the two. A context-preserving mechanism, either Lossless Forwarding or Adaptive Scaling, restores the BiLSTM context that the non-linear step would otherwise wash out. Finally, a position-aware CNN with max pooling extracts n-gram features for classification. On several benchmarks, TNet outperforms earlier state-of-the-art models, and ablations support each component. This matters because it shows CNN feature extraction can replace attention for this task when combined with target-aware word transformation.

Key points

  • Two variants are evaluated, named after their context-preserving strategy: TNet-LF (Lossless Forwarding, residual-style addition) and TNet-AS (Adaptive Scaling, a learned gate mixing transformed and original features).
  • TST weights target words by their relatedness to each sentence word using a softmax over dot products. As a result, a word like "long" can be read differently for "battery life" than for "startup times".
  • Varying the number of CPT layers on held-out LAPTOP training data showed both variants peaked at L=2, with performance generally dropping as L grew.
  • In qualitative comparisons, TNet handled contrastive sentences better than RAM and BILSTM-ATT-G, partly via size-three convolution kernels that capture informative trigrams. All models still failed on an implicit, subjunctive-style example.
  • The code is open source at github.com/lixin4ever/TNet.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

81 papers of our library cite it (of the 140884 whose bibliographies we read); 7 of them have a page here, 74 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p18-1087

@inproceedings{li2018transformation,
  title = {{Transformation Networks for Target-Oriented Sentiment Classification}},
  author = {Li, Xin and Bing, Lidong and Lam, Wai and Shi, Bei},
  booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2018},
  doi = {10.18653/v1/p18-1087},
  url = {https://doi.org/10.18653/v1/p18-1087}
}

References

37 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly Xin Li and Wai Lam. 2017. Deep multi-task learning for aspect term extraction with memory interaction. In Proceedings of EMNLP, pages 2876–2882. learning to align and translate. In Proceedings of ICLR. find
  2. Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8):1798–1828. find our card
  3. Bing Liu. 2012. Sentiment analysis and opinion mining. Synthesis Lectures on Human Language Technologies, 5(1):1–167. find
  4. Jiangming Liu and Yue Zhang. 2017. Attention modeling for targeted sentiment. In Proceedings of EACL, pages 572–577. find
  5. Sasha Blair-Goldensohn, Kerry Hannan, Ryan McDonald, Tyler Neylon, George A Reis, and Jeff Reynar. 2008. Building a sentiment summarizer for local service reviews. In WWW workshop on NLP in the information explosion era, pages 339–348. find
  6. Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of EMNLP, pages 1412–1421. find
  7. Peng Chen, Zhongqian Sun, Lidong Bing, and Wei Yang. 2017. Recurrent attention network on memory for aspect sentiment analysis. In Proceedings of EMNLP, pages 463–472. find our card
  8. Dehong Ma, Sujian Li, Xiaodong Zhang, and Houfeng Wang. 2017. Interactive attention networks for aspect-level sentiment classification. In Proceedings of IJCAI, pages 4068–4074. find
  9. Li Dong, Furu Wei, Chuanqi Tan, Duyu Tang, Ming Zhou, and Ke Xu. 2014. Adaptive recursive neural network for target-dependent twitter sentiment classification. In Proceedings of ACL, pages 49–54. find our card
  10. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of EMNLP, pages 1532–1543. find
  11. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016a. Deep residual learning for image recognition. In Proceedings of CVPR, pages 770–778. find
  12. Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. Semeval-2014 task 4: Aspect based sentiment analysis. In Proceedings of SemEval, pages 27–35. find our card
  13. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016b. Identity mappings in deep residual networks. In Proceedings of ECCV, pages 630–645. find
  14. Long Jiang, Mo Yu, Ming Zhou, Xiaohua Liu, and Tiejun Zhao. 2011. Target-dependent twitter sentiment classification. In Proceedings of ACL, pages 151–160. find
  15. Bei Shi, Zihao Fu, Lidong Bing, and Wai Lam. 2018. Learning domain-sensitive and sentiment-aware word embeddings. In Proceedings of ACL. find
  16. Rupesh K Srivastava, Klaus Greff, and Jürgen Schmidhuber. 2015a. Training very deep networks. In Proceedings of NIPS, pages 2377–2385. find
  17. Rie Johnson and Tong Zhang. 2015. Semi-supervised convolutional neural networks for text categorization via region embedding. In Proceedings of NIPS, pages 919–927. find
  18. Rupesh Kumar Srivastava, Klaus Greff, and Jürgen Schmidhuber. 2015b. Highway networks. arXiv preprint arXiv:1505.00387. find our card
  19. Rafal Jozefowicz, Wojciech Zaremba, and Ilya Sutskever. 2015. An empirical exploration of recurrent network architectures. In Proceedings of ICML, pages 2342–2350. find
  20. Duyu Tang, Bing Qin, Xiaocheng Feng, and Ting Liu. 2016a. Effective lstms for target-dependent sentiment classification. In Proceedings of COLING, pages 3298–3307. find our card
  21. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of EMNLP, pages 1746–1751. find our card
  22. Duyu Tang, Bing Qin, and Ting Liu. 2016b. Aspect level sentiment classification with deep memory network. In Proceedings of EMNLP, pages 214–224. find our card
  23. Diederik Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of ICLR. find
  24. Yi Tay, Anh Tuan Luu, and Siu Cheung Hui. 2017. Learning to attend via word-aspect associative fusion for aspect-based sentiment analysis. arXiv preprint arXiv:1712.05403. — [arXiv:1712.05403](https://arxiv.org/abs/1712.05403) find
  25. Svetlana Kiritchenko, Xiaodan Zhu, Colin Cherry, and Saif Mohammad. 2014. Nrc-canada-2014: Detecting aspects and sentiment in customer reviews. In Proceedings of SemEval, pages 437–442. find our card
  26. Ivan Titov and Ryan McDonald. 2008. Modeling online reviews with multi-grain topic models. In Proceedings of WWW, pages 111–120. find
  27. Siwei Lai, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Recurrent convolutional neural networks for text classification. In Proceedings of AAAI, volume 333, pages 2267–2273. find
  28. Duy-Tin Vo and Yue Zhang. 2015. Target-dependent twitter sentiment classification with rich automatic features. In Proceedings of IJCAI, pages 1347– 1353. find
  29. Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature, 521(7553):436. find
  30. Joachim Wagner, Piyush Arora, Santiago Cortes, Utsab Barman, Dasha Bogdanova, Jennifer Foster, and Lamia Tounsi. 2014. Dcu: Aspect-based polarity classification for semeval task 4. In Proceedings of SemEval, pages 223–229. find
  31. Yequan Wang, Minlie Huang, xiaoyan zhu, and Li Zhao. 2016. Attention-based lstm for aspect-level sentiment classification. In Proceedings of EMNLP, pages 606–615. find our card
  32. Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhudinov, Rich Zemel, and Yoshua Bengio. 2015. Show, attend and tell: Neural image caption generation with visual attention. In Proceedings of ICML, pages 2048–2057. find our card
  33. Min Yang, Wenting Tu, Jingxuan Wang, Fei Xu, and Xiaojun Chen. 2017. Attention based lstm for target dependent sentiment classification. In Proceedings of AAAI, pages 5013–5014. find
  34. Jianxing Yu, Zheng-Jun Zha, Meng Wang, and Tat-Seng Chua. 2011. Aspect ranking: identifying important product aspects from online consumer reviews. In Proceedings of ACL, pages 1496–1505. find
  35. Meishan Zhang, Yue Zhang, and Duy-Tin Vo. 2016. Gated neural networks for targeted sentiment analysis. In Proceedings of AAAI, pages 3087–3093. find
  36. Ye Zhang and Byron Wallace. 2017. A sensitivity analysis of (and practitioners guide to) convolutional neural networks for sentence classification. In Proceedings of IJCNLP, pages 253–263. find our card
  37. Yue Zhang, Zhenghua Li, Jun Lang, Qingrong Xia, and Min Zhang. 2017. Dependency parsing with partial annotations: An empirical comparison. In Proceedings of IJCNLP, pages 49–58. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/li-2018-transformation-networks-target-oriented-sentiment/