Elamur Research⌕
☰
home / papers / 2020 / Avoiding the Hypothesis-Only Bias in Natural Language Inference via Ensemble Adversarial Training

Avoiding the Hypothesis-Only Bias in Natural Language Inference via Ensemble Adversarial Training

Paper · 2020 · Joe Stacey, Pasquale Minervini, Haim Dubossarsky et al.

Summary

NLI datasets such as SNLI contain annotation artefacts. Negation words, for example, tend to appear in contradiction hypotheses, so a model can guess the label from the hypothesis alone and then transfer poorly to other data. Earlier work by Belinkov et al. trained a single hypothesis-only adversary against a shared sentence encoder. However, the bias stayed in the representations: once the encoder was frozen, a new classifier could largely relearn it.

Stacey et al. extend this setup to an ensemble of n hypothesis-only adversaries. The encoder and the premise–hypothesis classifier fit the data while jointly pushing down the accuracy of every adversary. Bias is measured by freezing the representations and training a fresh classifier to recover it.

With enough dimensions, the ensemble removes significantly more bias than one adversary. Larger representations are harder to clean and benefit from more adversaries. The de-biased models also generalise better across 12 other NLI datasets, beating both single-adversary training and the Product of Experts approach of Mahabadi et al. (2020). This links an actual drop in stored bias to better robustness.

Key points

  • On SNLI, hypothesis-only models reach about 67% accuracy against a 34% majority baseline. Known artefacts include negation words and hypothesis length, with entailment hypotheses tending to be shorter.
  • Relearned-bias accuracy, single adversary vs 5 adversaries:
    • 512 dims: 49.8 vs 45.8
    • 2,048 dims: 61.6 vs 58.7
    • The reductions are significant (Bonferroni-corrected Mann-Whitney and bootstrap tests) for 512 dimensions and above, but not at 256.
  • On SNLI-hard, one adversary gives +1.1 points (p=0.07, not significant), while 20 adversaries give +1.6 points (p=0.015).
  • Average gains over the baseline across the evaluation sets: PoE +1.2, one adversary +1.6, ensemble +2.5. The ensemble beats one adversary on 8 of 13 datasets and ties on 3.
  • A non-linear MLP adversary is not always better. The best adversary type depends on whether the classifier that later tries to relearn the bias is linear or non-linear. Removing bias as seen by a non-linear probe remains an open problem.

Interesting ideas

  • I1. Freeze and re-probe. After de-biasing, freeze the representations and train a fresh probe to see whether the unwanted signal can still be recovered, instead of trusting the adversary's own low accuracy.
  • I2. Partial-input baselines. Measure how well a task can be solved from only part of the input (hypothesis only, candidate ending only, last paragraph only) to expose dataset artefacts.
  • I3. Hard subsets. Define a "hard" subset as the examples a partial-input model gets wrong, and report scores on it separately.
  • I4. Scale adversaries with capacity. The number of adversaries needed grows with representation size, so de-biasing effort should match model capacity rather than stay fixed.
  • I5. Match the adversary to the reader. The adversary should match the classifier that will actually consume the representation (linear vs non-linear); a stronger adversary is not automatically better.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Use this as a negative control in evaluation, checking whether a shortcut feature such as sentence length or a cue word can still be read from learned features after any "debiased" training.
  • I2 (answer): Run each school-test and fable-QA set with the story removed; any above-chance score marks questions that cannot test real understanding.
  • I3 (learn): Split benchmarks into items a shallow cue model solves and items it fails, and track progress mainly on the latter to see whether the explainable pipeline adds value.
  • I4 (tree): When training gparse with an adversarial head that hides treebank or UD-dialect identity, scale the number of adversaries with hidden size and verify the result with a frozen probe.
  • I5 (learn): Pick adversary or probe architectures that mirror the downstream consumer, such as the perceptron parser or a linear role scorer.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of CambridgeCambridge · United Kingdom155 works · 115,562 citations🎓University College LondonLondon · United Kingdom59 works · 16,463 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 510 works

Cite

Cite as Joe Stacey, Pasquale Minervini, Haim Dubossarsky, Sebastian Riedel and Tim Rocktäschel. 2020. Avoiding the Hypothesis-Only Bias in Natural Language Inference via Ensemble Adversarial Training. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.emnlp-main.665

@inproceedings{stacey2020avoiding,
  title = {{Avoiding the Hypothesis-Only Bias in Natural Language Inference via Ensemble Adversarial Training}},
  author = {Joe Stacey, Pasquale Minervini, Haim Dubossarsky, Sebastian Riedel and Tim Rocktäschel},
  booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.665},
  url = {https://doi.org/10.18653/v1/2020.emnlp-main.665}
}

References

52 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aishwarya Agrawal, Dhruv Batra, Devi Parikh, and Aniruddha Kembhavi. 2018. Don’t just assume; look and answer: Overcoming priors for visual question answering. In CVPR, pages 4971–4980. IEEE Computer Society. find
  2. Yonatan Belinkov, Adam Poliak, Stuart M. Shieber, Benjamin Van Durme, and Alexander M. Rush. 2019a. Don’t take the premise for granted: Mitigating artifacts in natural language inference. In ACL (1), pages 877–891. Association for Computational Linguistics.
  3. Yonatan Belinkov, Adam Poliak, Stuart M. Shieber, Benjamin Van Durme, and Alexander M. Rush. 2019b. On adversarial removal of hypothesis-only bias in natural language inference. In *SEM@NAACL-HLT, pages 256–262. Association for Computational Linguistics.
  4. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In EMNLP, pages 632–642. The Association for Computational Linguistics. ↗
  5. Zheng Cai, Lifu Tu, and Kevin Gimpel. 2017. Pay attention to the ending: Strong neural baselines for the ROC story cloze task. In ACL (2), pages 616–622. Association for Computational Linguistics. find
  6. Christopher Clark, Mark Yatskar, and Luke Zettlemoyer. 2019. Don’t take the easy way out: Ensemble based methods for avoiding known dataset biases. In EMNLP-IJCNLP, pages 4060–4073. Association for Computational Linguistics. ↗
  7. Alexis Conneau, Douwe Kiela, Holger Schwenk, Loı̈c Barrault, and Antoine Bordes. 2017. Supervised learning of universal sentence representations from natural language inference data. In EMNLP, pages 670–680. Association for Computational Linguistics.
  8. Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2018. Hotflip: White-box adversarial examples for text classification. In ACL (2), pages 31–36. Association for Computational Linguistics.
  9. Bradley Efron and Robert Tibshirani. 1993. An Introduction to the Bootstrap. Springer. ↗
  10. Yanai Elazar and Yoav Goldberg. 2018. Adversarial removal of demographic attributes from text data. In EMNLP, pages 11–21. Association for Computational Linguistics.
  11. Yaroslav Ganin and Victor S. Lempitsky. 2015. Unsupervised domain adaptation by backpropagation. In ICML, volume 37 of JMLR Workshop and Conference Proceedings, pages 1180–1189. JMLR.org. ↗
  12. Ian J. Goodfellow, Yoshua Bengio, and Aaron C. Courville. 2016. Deep Learning. Adaptive computation and machine learning. MIT Press. find
  13. Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. 2017. Making the V in VQA matter: Elevating the role of image understanding in visual question answering. In CVPR, pages 6325–6334. IEEE Computer Society. ↗
  14. Edward Grefenstette, Robert Stanforth, Brendan O’Donoghue, Jonathan Uesato, Grzegorz Swirszcz, and Pushmeet Kohli. 2018. Strength in numbers: Trading-off robustness and compuputation via adversarially-trained ensembles. find
  15. Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel R. Bowman, and Noah A. Smith. 2018. Annotation artifacts in natural language inference data. In NAACL-HLT (2), pages 107–112. Association for Computational Linguistics. find
  16. He He, Sheng Zha, and Haohan Wang. 2019. Unlearn dataset bias in natural language inference by fitting the residual. In DeepLo@EMNLP-IJCNLP.
  17. Felix Hill, Antoine Bordes, Sumit Chopra, and Jason Weston. 2016. The goldilocks principle: Reading children’s books with explicit memory representations. In ICLR. ↗
  18. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735–1780. ↗
  19. Hossein Hosseini, Baicen Xiao, and Radha Poovendran. 2017. Deceiving google’s cloud video intelligence API built for summarizing videos. In CVPR Workshops, pages 1305–1309. IEEE Computer Society. find
  20. Mohit Iyyer, John Wieting, Kevin Gimpel, and Luke Zettlemoyer. 2018. Adversarial example generation with syntactically controlled paraphrase networks. In NAACL-HLT, pages 1875–1885. Association for Computational Linguistics.
  21. Robin Jia and Percy Liang. 2017. Adversarial examples for evaluating reading comprehension systems. In EMNLP, pages 2021–2031. Association for Computational Linguistics.
  22. Kushal Kafle and Christopher Kanan. 2016. Answer-type prediction for visual question answering. In CVPR, pages 4976–4984. IEEE Computer Society. find
  23. Divyansh Kaushik and Zachary C. Lipton. 2018. How much reading does reading comprehension require? A critical investigation of popular benchmarks. In EMNLP, pages 5010–5015. Association for Computational Linguistics.
  24. Tushar Khot, Ashish Sabharwal, and Peter Clark. 2018. Scitail: A textual entailment dataset from science question answering. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, pages 5189–5197. AAAI Press. find
  25. Alice Lai, Yonatan Bisk, and Julia Hockenmaier. 2017. Natural language inference from multiple premises. In Proceedings of the Eighth International Joint Conference on Natural Language Processing, IJC-NLP 2017, Taipei, Taiwan, November 27 - December 1, 2017 - Volume 1: Long Papers, pages 100– 109. Asian Federation of Natural Language Processing. ↗
  26. Tianyi Lin, Chi Jin, and Michael I. Jordan. 2019. On gradient descent ascent for nonconvex-concave minimax problems. CoRR, abs/1906.00331. ↗
  27. Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. 2014. A SICK cure for the evaluation of compositional distributional semantic models. In LREC, pages 216–223. European Language Resources Association (ELRA).
  28. Pasquale Minervini and Sebastian Riedel. 2018. Adversarially regularising neural NLI models to integrate logical background knowledge. In CoNLL, pages 65–74. Association for Computational Linguistics. ↗
  29. Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James F. Allen. 2016. A corpus and cloze evaluation for deeper understanding of commonsense stories. In HLT-NAACL, pages 839– 849. The Association for Computational Linguistics. find
  30. Ellie Pavlick and Chris Callison-Burch. 2016. Most ”babies” are ”little” and most ”problems” are ”huge”: Compositional entailment in adjective-nouns. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers. The Association for Computer Linguistics. find
  31. Ellie Pavlick, Travis Wolfe, Pushpendre Rastogi, Chris Callison-Burch, Mark Dredze, and Benjamin Van Durme. 2015. Framenet+: Fast paraphrastic tripling of framenet. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, ACL 2015, July 26-31, 2015, Beijing, China, Volume 2: Short Papers, pages 408–413. The Association for Computer Linguistics.
  32. Adam Poliak, Jason Naradowsky, Aparajita Haldar, Rachel Rudinger, and Benjamin Van Durme. 2018. Hypothesis only baselines in natural language inference. In *SEM@NAACL-HLT, pages 180–191. Association for Computational Linguistics. find
  33. Altaf Rahman and Vincent Ng. 2012. Resolving complex cases of definite pronouns: The winograd schema challenge. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLP-CoNLL 2012, July 12-14, 2012, Jeju Island, Korea, pages 777–789. ACL.
  34. Rabeeh Karimi Mahabadi, Yonatan Belinkov, and James Henderson. 2020. End-to-end bias mitigation by modelling biases in corpora. ↗
  35. Drew Reisinger, Rachel Rudinger, Francis Ferraro, Craig Harman, Kyle Rawlins, and Benjamin Van Durme. 2015. Semantic proto-roles. Trans. Assoc. Comput. Linguistics, 3:475–488. ↗
  36. Henry B Mann and Donald R Whitney. 1947. On a test of whether one of two random variables is stochastically larger than the other. In The annals of Mathematical Statisitcs, pages 50–60. Institute of Mathematical Statistics. ↗
  37. Marco Túlio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Semantically equivalent adversarial rules for debugging NLP models. In ACL (1), pages 856–865. Association for Computational Linguistics. find
  38. Roy Schwartz, Maarten Sap, Ioannis Konstas, Leila Zilles, Yejin Choi, and Noah A. Smith. 2017. The effect of different writing tasks on linguistic style: A case study of the ROC story cloze task. In CoNLL, pages 15–25. Association for Computational Linguistics. find
  39. Juliet Popper Shaffer. 1995. Multiple hypothesis testing. Annual Review of Psychology, 46:561–581. ↗
  40. Rion Snow, Lucy Vanderwende, and Arul Menezes. 2006. Effectively using syntax for recognizing false entailment. In HLT-NAACL. The Association for Computational Linguistics. find
  41. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. 2014. Intriguing properties of neural networks. In ICLR (Poster). pages 996–1005. Asian Federation of Natural Language Processing. ↗
  42. Adina Williams, Nikita Nangia, and Samuel R. Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In NAACL-HLT, pages 1112–1122. Association for Computational Linguistics. find
  43. Peng Zhang, Yash Goyal, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. 2016. Yin and yang: Balancing and answering binary visual questions. In CVPR, pages 5014–5022. IEEE Computer Society. find
  44. Sheng Zhang, Rachel Rudinger, Kevin Duh, and Benjamin Van Durme. 2017. Ordinal commonsense inference. Transactions of the Association for Computational Linguistics, 5:379–395. find
  45. Aarne Talman and Stergios Chatzikyriakidis. 2018. Neural network models for natural language inference fail to capture the semantics of inference. CoRR, abs/1810.09774. ↗
  46. Masatoshi Tsuchiya. 2018. Performance impact caused by hidden bias of training data for recognizing textual entailment. In LREC. European Language Resources Association (ELRA).
  47. Lucy Vanderwende and William B. Dolan. 2005. What syntax can contribute in the entailment task. In MLCW, volume 3944 of Lecture Notes in Computer Science, pages 205–216. Springer. find
  48. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the Workshop: Analyzing and Interpreting Neural Networks for NLP, BlackboxNLP@EMNLP 2018, Brussels, Belgium, November 1, 2018, pages 353–355. Association for Computational Linguistics. ↗
  49. William Yang Wang, Sameer Singh, and Jiwei Li. 2019. Deep adversarial learning for NLP. In NAACL-HLT (Tutorial Abstracts), pages 1–5. Association for Computational Linguistics. find
  50. Yicheng Wang and Mohit Bansal. 2018. Robust machine comprehension models via adversarial training. In NAACL-HLT (2), pages 575–581. Association for Computational Linguistics. ↗
  51. Jason Weston, Antoine Bordes, Sumit Chopra, and Tomas Mikolov. 2016. Towards ai-complete question answering: A set of prerequisite toy tasks. In ICLR (Poster).
  52. Aaron Steven White, Pushpendre Rastogi, Kevin Duh, and Benjamin Van Durme. 2017. Inference is everything: Recasting semantic resources into a unified evaluation framework. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers),

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/stacey-2020-avoiding-hypothesis-only-bias-natural/