Elamur Research⌕
☰
home / papers / 2024 / It is not True that Transformers are Inductive Learners: Probing NLI Models with External Negation

It is not True that Transformers are Inductive Learners: Probing NLI Models with External Negation

Paper · 2024 · Michael Sullivan

Summary

The paper asks whether transformer models fine-tuned for natural language inference actually reason logically or only seem to. It uses external negation as the test case. Prefixing a hypothesis with a clause like "it is not true that" changes the gold label in a fully predictable way: entailment becomes contradiction, contradiction becomes entailment, and neutral stays neutral. This lets the author build challenge sets from MNLI and SNLI automatically.

Six encoder and encoder-decoder models (including RoBERTa, DeBERTa and BART variants) are tested with the inoculation-by-fine-tuning paradigm. Three experiments cover:

  • stacking the prefix to depths not seen in training;
  • switching to an unseen but similar prefix;
  • diagnosing why that transfer fails.

All six models initially behave as if the negation prefix were noise. DeBERTa and BART never learn the excluded-middle pattern. RoBERTa learns it for one prefix but cannot carry it over to "it is false that", because inoculation erases its prior sense that the two prefixes are similar.

A formal result (Theorem 1) shows that an encoder transformer could, in principle, model this behavior. The failure therefore points to training data and procedure, not to the architecture itself.

Key points

  • Label rule: the augmented example (P, ¬H) maps entailment to contradiction and contradiction to entailment, and keeps neutral unchanged. No human relabeling is needed.
  • Datasets: challenge data comes only from MNLI and SNLI. ANLI was dropped because every model already did poorly on its original test set.
  • Experiment 1: after standard NLI fine-tuning, all six models ignore "it is not true that". DeBERTa and BART fail to generalize double-negation cancellation to longer prefix chains even after extensive inoculation.
  • Experiments 2–3: the three RoBERTa models handle the trained prefix but collapse on "it is false that". The cause is catastrophic forgetting of how the two prefixes relate.
  • Theorem 1: stated in the logic FOC[+;MOD] for fixed-precision encoders with a fixed input length. Decoder-only models such as LLaMa-2 and GPT-3 are not studied. Code is at github.com/mjs227/AdversarialNLI.

Interesting ideas

  • I1. Build challenge items with a transformation whose effect on the label is known by rule, so large probe sets need no annotators
  • I2. Test for generalization by nesting the same operator to depths beyond those seen in training, rather than only adding more examples
  • I3. After teaching one surface form, probe a near-synonym of it to catch forgetting or over-narrow learning
  • I4. Use inoculation (fine-tune on the challenge set, then re-test both the original and challenge sets) to tell whether a failure comes from the data or from the model
  • I5. Separate whether a model can represent a function at all (a proof) from whether its training actually reaches it (experiments)

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): wrap hypotheses in external negation and check that the answer flips exactly as logic predicts, giving free negative controls for evaluation.
  • I2 (world): stack "it is not true that" layers and verify that the world-state truth value alternates correctly, exposing whether negation is composed or merely pattern-matched.
  • I3 (frames): check that "it is false that" and "it is not true that" map to the same negation frame, so that new repair rules for one form do not silently break the other.
  • I4 (learn): apply this when gparse or the tagger fails on a construction, to decide between adding silver data and changing the model design.
  • I5 (answer): when a reader fails on negated questions, first confirm the pipeline can express the operation in principle, then look at training or rules as the bottleneck.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University at Buffalo, State University of New YorkBuffalo · United States12 works · 318 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Reasoning · 1,426 worksInterpretability · 130 works

Cite

Cite as Michael Sullivan. 2024. It is not True that Transformers are Inductive Learners: Probing NLI Models with External Negation. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.eacl-long.116

@inproceedings{sullivan2024it,
  title = {{It is not True that Transformers are Inductive Learners: Probing NLI Models with External Negation}},
  author = {Michael Sullivan},
  booktitle = {Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2024},
  doi = {10.18653/v1/2024.eacl-long.116},
  url = {https://doi.org/10.18653/v1/2024.eacl-long.116}
}

References

33 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Akari Asai and Hannaneh Hajishirzi. 2020. Logic-guided data augmentation and regularization for consistent question answering. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5642–5650, Online. Association for Computational Linguistics.
  2. Yonatan Belinkov and James Glass. 2019. Analysis methods in neural language processing: A survey. Transactions of the Association for Computational Linguistics, 7:49–72. ↗
  3. Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150. ↗
  4. Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015. A large annotated corpus for learning natural language inference. arXiv preprint arXiv:1508.05326. ↗
  5. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran Associates, Inc.
  6. David Chiang, Peter Cholak, and Anand Pillay. 2023. Tighter bounds on the expressivity of transformer encoders. In Proc. ICML. ↗
  7. Tiffany Chien and Jugal Kalita. 2020. Adversarial analysis of natural language inference systems. In 2020 IEEE 14th International Conference on Semantic Computing (ICSC), pages 1–8. IEEE.
  8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  9. Allyson Ettinger. 2020. What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models. Transactions of the Association for Computational Linguistics, 8:34–48. ↗
  10. Yaroslav Fyodorov, Yoad Winter, and Nissim Francez. 2000. A natural logic inference system. In Proceedings of the 2nd Workshop on Inference in Computational Semantics (ICoS-2). find
  11. Ivan Habernal, Henning Wachsmuth, Iryna Gurevych, and Benno Stein. 2018. SemEval-2018 task 12: The argument reasoning comprehension task. In Proceedings of the 12th International Workshop on Semantic Evaluation, pages 763–772, New Orleans, Louisiana. Association for Computational Linguistics. find
  12. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020. Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654. ↗
  13. Md Mosharaf Hossain, Venelin Kovatchev, Pranoy Dutta, Tiffany Kao, Elizabeth Wei, and Eduardo Blanco. 2020. An analysis of natural language inference benchmarks through the lens of negation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9106–9118, Online. Association for Computational Linguistics. ↗
  14. Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, R Devon Hjelm, Alessandro Sordoni, and Aaron Courville. 2021. Understanding by understanding not: Modeling negation in language models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1301–1312, Online. Association for Computational Linguistics. ↗
  15. Paul Hovda. 2009. What is classical mereology? Journal of Philosophical Logic, 38:55–82. ↗
  16. Neil Immerman. 2012. Descriptive complexity. Springer Science & Business Media. find
  17. Myeongjun Jang, Deuk Sin Kwon, and Thomas Lukasiewicz. 2022. BECEL: Benchmark for consistency evaluation of language models. In Proceedings of the 29th International Conference on Computational Linguistics, pages 3680–3696, Gyeongju, Republic of Korea. International Committee on Computational Linguistics. find
  18. Nora Kassner and Hinrich Schütze. 2020. Negated and misprimed probes for pretrained language models: Birds can talk, but cannot fly. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7811–7818, Online. Association for Computational Linguistics.
  19. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. ↗
  20. Antonio Laverghetta Jr. and John Licato. 2022. Developmental negation processing in transformer language models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 545–551, Dublin, Ireland. Association for Computational Linguistics. find
  21. Antonio Laverghetta Jr., Animesh Nighojkar, Jamshidbek Mirzakhalov, and John Licato. 2021. Can transformer language models predict psychometric properties? In Proceedings of *SEM 2021: The Tenth Joint Conference on Lexical and Computational Semantics, pages 12–25, Online. Association for Computational Linguistics. find
  22. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871–7880, Online. Association for Computational Linguistics. ↗
  23. Nelson F. Liu, Roy Schwartz, and Noah A. Smith. 2019a. Inoculation by fine-tuning: A method for analyzing challenge datasets. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2171–2179, Minneapolis, Minnesota. Association for Computational Linguistics.
  24. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. ↗
  25. Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3428–3448, Florence, Italy. Association for Computational Linguistics. ↗
  26. Aakanksha Naik, Abhilasha Ravichander, Norman Sadeh, Carolyn Rose, and Graham Neubig. 2018. Stress test evaluation for natural language inference. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2340–2353, Santa Fe, New Mexico, USA. Association for Computational Linguistics. ↗
  27. Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020. Adversarial NLI: A new benchmark for natural language understanding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4885–4901, Online. Association for Computational Linguistics. ↗
  28. Timothy Niven and Hung-Yu Kao. 2019. Probing neural network comprehension of natural language arguments. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4658–4664, Florence, Italy. Association for Computational Linguistics. find
  29. Kyle Richardson, Hai Hu, Lawrence Moss, and Ashish Sabharwal. 2020. Probing natural language inference models through semantic fragments. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 8713–8721. ↗
  30. Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2020. A primer in BERTology: What we know about how BERT works. Transactions of the Association for Computational Linguistics, 8:842–866. ↗
  31. James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: a large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 809–819, New Orleans, Louisiana. Association for Computational Linguistics. Formulas of FOC[+;MOD] are defined as one of: Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288. • ⊤ (true) or ⊥ (false). • Q a (p), where a ∈ Σ, and Q a (p) := σ p = a • M OD b a (p), where a ≥ 0, b > 0, and p is a position variable; M OD b a (p) := p ≡ b a Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30. • ϕ ∧ ψ, ϕ ∨ ψ, or ¬ψ, where ϕ and ψ are formulas. 11 • x 1 = x 2 or x 1 < x 2 , where x 1 , x 2 are in the sort of counts. 12 ↗
  32. Adina Williams, Nikita Nangia, and Samuel R Bowman. 2017. A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426. ↗
  33. Zhangdie Yuan, Songbo Hu, Ivan Vulić, Anna Korhonen, and Zaiqiao Meng. 2023. Can pretrained language models (yet) reason deductively? In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1447–1462, Dubrovnik, Croatia. Association for Computational Linguistics. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sullivan-2024-it-is-not-true-that/