Elamur Research
☰
home / papers / 2016 / Visualizing and Understanding Neural Models in NLP

Visualizing and Understanding Neural Models in NLP

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/N16-1082.pdf

Summary

Neural NLP models often outperform feature-based systems, but their vector representations are hard to interpret. In particular, it is unclear how they compose sentence meaning from words, for example when handling negation, intensification or contrasting clauses. Drawing on interpretation techniques from computer vision, the authors propose several visualization strategies.

They train a standard tanh RNN, an LSTM and a Bidirectional LSTM on the Stanford Sentiment Treebank. They also train an LSTM SEQ2SEQ autoencoder on English sentences from WMT'14. Their analysis tools are:

  • heatmaps and t-SNE plots of the composed representations,
  • a saliency score based on first-order derivatives, measuring how much each word contributes to the output,
  • a variance-based measure that scores a word by how far its embedding lies from the sentence average.

The plots reveal the known markedness asymmetry of negation and show that a few dimensions behave in a fairly localist way. Saliency maps suggest that LSTMs focus more sharply on sentiment-bearing words than plain RNNs do. The autoencoder appears to learn input–output alignments on its own. These general-purpose tools offer an early step toward explaining how deep networks build meaning.

Key points

  • Data: The Stanford Sentiment Treebank provides labels for 215,154 phrases in 11,855 sentences. Parse-tree nodes were converted to token sequences and classified at fine- and coarse-grained levels.

  • Sentence-level accuracy (60-dimensional embeddings and hidden layers, dropout 0.1, AdaGrad):

    ModelFine-grainedCoarse-grained
    Standard RNN0.4290.850
    LSTM0.4690.870
    Bi-LSTM0.4880.878

  • Negation asymmetry: In the t-SNE plots, "not bad" sits closer to negative words than to positive ones. This suggests the model does not simply apply one fixed "not" transformation.

  • Autoencoder behaviour: The SEQ2SEQ autoencoder was trained on 4 million WMT'14 English sentences averaging 22.5 words. As decoding proceeds, the source tokens lose influence and previously generated words, acting like a language model, take over.

  • Limits of the methods: Saliency could not account for why the plain RNN correctly classified an example sentence in which the speaker says they hate the film while granting that its storyline is interesting. The variance method only works when embeddings are learned from scratch.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

217 papers of our library cite it (of the 140673 whose bibliographies we read); 14 of them have a page here, 203 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n16-1082

@inproceedings{li2016visualizing,
  title = {{Visualizing and Understanding Neural Models in NLP}},
  author = {Li, Jiwei and Chen, Xinlei and Hovy, Eduard and Jurafsky, Dan},
  booktitle = {Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year = {2016},
  doi = {10.18653/v1/n16-1082},
  url = {https://doi.org/10.18653/v1/n16-1082}
}

References

29 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Figure 9: Saliency heatmap for S EQ 2S EQ auto-encoder in terms of predicting correspondent token at each time step.* different neural models. \#\# Acknowledgement The authors want to thank Sam Bowman, Percy Liang, Will Monroe, Sida Wang, Chris Manning and other members of the Stanford NLP group, as w… find
  2. Harcourt Brace Jovanovich. Navneet Dalal and Bill Triggs. 2005. Histograms of oriented gradients for human detection. In Computer Vision and Pattern Recognition, 2005. find
  3. CVPR 2005. IEEE Computer Society Conference on, volume 1, pages 886– 893. find
  4. IEEE. Jeffrey L. Elman. 1989. Representation and structure in connectionist models. Technical Report 8903, Center for Research in Language, University of California, San Diego. find
  5. Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2009. Visualizing higher-layer features of a deep network. Dept. IRO, Université de Montréal, Tech. find
  6. Rep. Manaal Faruqui and Chris Dyer. 2014. Improving vector space word representations using multilingual correlation. In Proceedings of EACL, volume 2014. find
  7. Manaal Faruqui, Yulia Tsvetkov, Dani Yogatama, Chris Dyer, and Noah Smith. 2015. Sparse overcomplete word vector representations. arXiv preprint arXiv:1506.02004. find
  8. Tamar Fraenkel and Yaacov Schul. 2008. The meaning of negated adjectives. Intercultural Pragmatics, 5(4):517– 540. find
  9. Alona Fyshe, Leila Wehbe, Partha P Talukdar, Brian Murphy, and Tom M Mitchell. 2015. A compositional and interpretable semantic space. Proceedings of the NAACL-HLT, Denver, USA. find
  10. Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation. In Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pages 580–587. IEEE. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Lon… find our card
  11. Laurence R. Horn. 1989. A natural history of negation, volume 960. University of Chicago Press Chicago. find
  12. Yangfeng Ji and Jacob Eisenstein. 2014. Representation learning for text-level discourse parsing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, volume 1, pages 13–24. find our card
  13. Andrej Karpathy, Justin Johnson, and Fei-Fei Li. 2015. Visualizing and understanding recurrent networks. arXiv preprint arXiv:1506.02078. find our card
  14. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105. find
  15. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2015. A diversity-promoting objective function for neural conversation models. arXiv preprint arXiv:1510.03055. find our card
  16. David G Lowe. 2004. Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60(2):91–110. find
  17. Thang Luong, Ilya Sutskever, Quoc V Le, Oriol Vinyals, and Wojciech Zaremba. 2014. Addressing the rare word problem in neural machine translation. arXiv preprint arXiv:1410.8206. find our card
  18. Aravindh Mahendran and Andrea Vedaldi. 2014. Understanding deep image representations by inverting them. arXiv preprint arXiv:1412.0035. — [arXiv:1412.0035](https://arxiv.org/abs/1412.0035) find
  19. Brian Murphy, Partha Pratim Talukdar, and Tom M Mitchell. 2012. Learning effective and interpretable semantic models using non-negative sparse embedding. In COLING, pages 1933–1950. find
  20. Anh Nguyen, Jason Yosinski, and Jeff Clune. 2014. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. arXiv preprint arXiv:1412.1897. — [arXiv:1412.1897](https://arxiv.org/abs/1412.1897) find
  21. Mike Schuster and Kuldip K Paliwal. 1997. Bidirectional recurrent neural networks. Signal Processing, IEEE Transactions on, 45(11):2673–2681. find
  22. Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2013. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034. find our card
  23. Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the conference on empirical methods in natural language processing (EMNLP), volum… find
  24. Citeseer. Ilya Sutskever, Oriol Vinyals, and Quoc VV Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems, pages 3104–3112. find
  25. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199. find our card
  26. Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of Machine Learning Research, 9(2579-2605):85. find
  27. Oriol Vinyals and Quoc Le. 2015. A neural conversational model. arXiv preprint arXiv:1506.05869. find our card
  28. Carl Vondrick, Aditya Khosla, Tomasz Malisiewicz, and Antonio Torralba. 2013. Hoggles: Visualizing object detection features. In Computer Vision (ICCV), 2013 IEEE International Conference on, pages 1–8. IEEE. Philippe Weinzaepfel, Hervé Jégou, and Patrick Pérez. 2011. Reconstructing an image from… find
  29. IEEE. Matthew D Zeiler and Rob Fergus. 2014. Visualizing and understanding convolutional networks. In Computer Vision–ECCV 2014, pages 818–833. Springer. find our card