Elamur Research⌕
☰
home / papers / 2018 / Behavior Analysis of NLI Models: Uncovering the Influence of Three Factors on Robustness

Behavior Analysis of NLI Models: Uncovering the Influence of Three Factors on Robustness

Paper · 2018 · Ivan Sanchez, Jeff Mitchell and Sebastian Riedel

Summary

High accuracy on SNLI says little about which behaviours NLI models have learned, or whether they lean on dataset artefacts. The authors use a behavioural-science approach. They treat three pre-trained SNLI models (a conditional encoder, DAM and ESIM) as subjects and test, with statistical tests, whether three factors shape their predictions:

  • insensitivity: the model fails to notice that a new instance differs from one it has seen;
  • polarity: a bias linking a word pair to a training label;
  • unseen pairs: word pairs never seen together in training.

To probe these factors, they take control instances from the SNLI training set that contain hypernym, hyponym or antonym pairs, swap the pair between premise and hypothesis, and annotate the result by hand where needed. Each model is scored on the control set and on the transformed set. All models stay robust when the label is unchanged but drop sharply when the swap changes the gold label. They often fall back on word–label correlations. They cope with unseen hypernym pairs but struggle with unseen antonyms. The work shows that factor-level behavioural analysis exposes weaknesses that a single accuracy score hides.

Key points

  • Models and SNLI dev accuracy:
    • conditional encoder: 0.782 (BiLSTMs, random 100-d embeddings);
    • DAM: 0.854 (frozen 300-d GloVe);
    • ESIM: 0.882.
  • Two settings for the transformed instances:
    • in situ: the original sentence context is kept, with the word pair swapped;
    • ex situ: premise and hypothesis share one sentence structure and differ by a single word.
  • Statistical tests:
    • chi-square tests (McNemar's, independence, homogeneity), with Yates's correction or Fisher's test for small samples;
    • significance threshold p < 0.0001 by default;
    • implemented with StatsModels and SciPy.
  • Label changes and accuracy:
    • In the ex situ hypernym/hyponym experiment, about 93% of transformed instances changed gold label.
    • Accuracy fell by 0.208 (CE), 0.061 (DAM) and 0.195 (ESIM).
    • On in situ instances whose label changed, CE and ESIM fell below chance, behaving much like nearest-neighbour lookup of the original training label.
  • Unseen pairs: Homogeneity tests found no link between errors and unseen hypernym pairs, which the authors attribute to hypernymy information in the embeddings. Unseen antonym pairs remained hard for all models.

Interesting ideas

  • I1. Build test items by minimally editing training instances, so every word and sentence frame was seen in training and any failure points to the factor under study rather than novelty.
  • I2. Treat the system as a behavioural subject: name candidate factors as independent variables and test their association with predictions using McNemar, independence or homogeneity tests rather than just reporting accuracy.
  • I3. Measure "polarity", i.e. how strongly a word pair co-occurs with one label in training, and compare accuracy on items where gold agrees versus disagrees with that prior to reveal shortcut learning.
  • I4. Learned representations handled hypernymy well but antonymy poorly, which the authors read as difficulty learning a symmetric relation from embeddings.
  • I5. The ex situ design builds premise and hypothesis from one sentence frame differing by one word, which isolates a single lexical relation as the only cue.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Swapping word pairs inside silver-data sentences could create negative-control sets that check whether a trained component truly flips its output when meaning flips.
  • I2 (learn): Evaluation reports could include significance tests for hypotheses such as "errors concentrate on unseen lemma pairs", turning vague error analysis into checked claims.
  • I3 (answer): For school tests and fable QA, item subsets could be split by whether surface cue words point to the right answer, to confirm the readers reason from the world model.
  • I4 (world): This supports compiling antonymy explicitly from WordNet into the knowledge layer instead of expecting it to emerge from vectors.
  • I5 (frames): Such one-word minimal pairs could test that the frames and semantic roles change only where the swapped word should affect them.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University College LondonLondon · United Kingdom59 works · 16,463 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 510 works

Cite

Cite as Ivan Sanchez, Jeff Mitchell and Sebastian Riedel. 2018. Behavior Analysis of NLI Models: Uncovering the Influence of Three Factors on Robustness. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n18-1179

@inproceedings{sanchez2018behavior,
  title = {{Behavior Analysis of NLI Models: Uncovering the Influence of Three Factors on Robustness}},
  author = {Ivan Sanchez, Jeff Mitchell and Sebastian Riedel},
  booktitle = {Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)},
  year = {2018},
  doi = {10.18653/v1/n18-1179},
  url = {https://doi.org/10.18653/v1/n18-1179}
}

References

30 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ethem Alpaydin. 2010. Introduction to Machine Learning. The MIT Press, 2nd edition. find
  2. Homa B. Hashemi and Rebecca Hwa. 2016. An evaluation of parser robustness for ungrammatical sentences. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, pages 1765–1774. https://aclweb.org/anthology/D16-1182. ↗
  3. Marco Baroni, Raffaella Bernardi, Ngoc-Quynh Do, and Chung-chieh Shan. 2012. Entailment above the word level in distributional semantics. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, Avignon, France, pages 23–32. http://www.aclweb.org/anthology/E12-1004. ↗
  4. Lucy P. Birkett and Nicholas E. Newton-Fisher. 2011. How abnormal is the behaviour of captive, zoo-living chimpanzees? PLOS ONE 6(6):1– 7. https://doi.org/10.1371/journal. pone.0020101. ↗
  5. Tolga Bolukbasi, Kai-Wei Chang, James Y. Zou, Venkatesh Saligrama, and Adam T. Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, Curran Associates, Inc., pages 4349–4357. ↗
  6. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Lisbon, Portugal, pages 632–642. http://aclweb.org/ anthology/D15-1075. ↗
  7. Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. 2017. Enhanced lstm for natural language inference. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Vancouver, Canada, pages 1657–1668. http://aclweb.org/anthology/P17-1152. ↗
  8. Mark Craven and Jude W. Shavlik. 1996. Extracting tree-structured representations of trained networks. In D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo, editors, Advances in Neural Information Processing Systems 8, MIT Press, pages 24–30. find
  9. Kate Crawford and Ryan Calo. 2016. There is a blind spot in ai research. Nature 538(7625). ↗
  10. Ido Dagan, Bill Dolan, Bernardo Magnini, and Dan Roth. 2009. Recognizing textual entailment: Rational, evaluation and approaches. Natural Language Engineering 15(4):i–xvii. https://doi.org/ 10.1017/S1351324909990209. ↗
  11. Ido Dagan and Oren Glickman. 2004. Probabilistic textual entailment: generic applied modeling of language variability. In PASCAL Workshop on Learning Methods for Text Understanding and Mining. Grenoble, France. find
  12. W. Frank Epling and W. David Pierce. 1986. The basic importance of applied behavior analysis. The Behavior Analyst 9(1):89–99. https://doi.org/ 10.1007/BF03391932. ↗
  13. Pierre Isabelle, Colin Cherry, and George Foster. 2017. A challenge set approach to evaluating machine translation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Copenhagen, Denmark, pages 2476–2486. https://www.aclweb.org/ anthology/D17-1262. ↗
  14. Robin Jia and Percy Liang. 2017. Adversarial examples for evaluating reading comprehension systems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Copenhagen, Denmark, pages 2011–2021. https://www.aclweb.org/anthology/D17-1214. ↗
  15. Jonathan K. Kummerfeld, David Hall, James R. Curran, and Dan Klein. 2012. Parser showdown at the wall street corral: An empirical investigation of error types in parser output. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Association for Computational Linguistics, Jeju Island, Korea, pages 1048–1059. http://www.aclweb.org/ anthology/D12-1096. ↗
  16. Jiwei Li, Will Monroe, and Dan Jurafsky. 2016. Understanding neural networks through representation erasure. CoRR abs/1612.08220. http://arxiv.org/abs/1612.08220. ↗
  17. Peter LoBue and Alexander Yates. 2011. Types of common-sense knowledge needed for recognizing textual entailment. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers - Volume 2. Association for Computational Linguistics, Stroudsburg, PA, USA, HLT ’11, pages 329–334. http://dl.acm.org/citation. cfm?id=2002736.2002805. ↗
  18. Bill Maccartney. 2009. Natural Language Inference. Ph.D. thesis, Stanford, CA, USA. AAI3364139. find
  19. J.H. McDonald. 2014. Handbook of Biological Statistics (3rd ed.). Sparky House Publishing, Baltimore, Maryland. find
  20. Joy Mench. 1998. Why it is important to understand animal behavior. ILAR Journal 39(1):20– 26. https://doi.org/10.1093/ilar.39. 1.20. Language Processing (Volume 1: Long Papers). Asian Federation of Natural Language Processing, Taipei, Taiwan, pages 996–1005. http://www. aclweb.org/anthology/I17-1100. ↗
  21. Saif M. Mohammad, Bonnie J. Dorr, Graeme Hirst, and Peter D. Turney. 2013. Computing lexical contrast. Computational Linguistics 39(3):555–590. ↗
  22. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Copenhagen, Denmark, pages 2979–2989. https://www.aclweb.org/anthology/D17-1323. ↗
  23. T. E. Oliphant. 2007. Python for scientific computing. Computing in Science Engineering 9(3):10– 20. https://doi.org/10.1109/MCSE. 2007.58. ↗
  24. Ankur Parikh, Oscar Täckström, Dipanjan Das, and Jakob Uszkoreit. 2016. A decomposable attention model for natural language inference. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, pages 2249–2255. https://aclweb.org/anthology/D16-1244. ↗
  25. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. ”why should i trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, NY, USA, KDD ’16, pages 1135–1144. https://doi.org/10.1145/ 2939672.2939778. ↗
  26. Mark Sammons, V. G. Vinod Vydiswaran, and Dan Roth. 2010. ”ask not what textual entailment can do for you...”. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Stroudsburg, PA, USA, ACL ’10, pages 1199–1208. http://dl.acm.org/ citation.cfm?id=1858681.1858803. ↗
  27. Ivan Sanchez and Sebastian Riedel. 2017. How well can we predict hypernyms from word embeddings? a dataset-centric analysis. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers. Association for Computational Linguistics, Valencia, Spain, pages 401–407. http://www.aclweb.org/ anthology/E17-2064. ↗
  28. Skipper Seabold and Josef Perktold. 2010. Statsmodels: Econometric and statistical modeling with python. In 9th Python in Science Conference. find
  29. Dilip Soman. 2001. Effects of payment mechanism on spending behavior: The role of rehearsal and immediacy of payments. Journal of Consumer Research 27(4):460–474. ↗
  30. Aaron Steven White, Pushpendre Rastogi, Kevin Duh, and Benjamin Van Durme. 2017. Inference is everything: Recasting semantic resources into a unified evaluation framework. In Proceedings of the Eighth International Joint Conference on Natural

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sanchez-2018-behavior-analysis-nli-models-uncovering/