Elamur Research⌕
☰
home / papers / 2022 / A Generative Approach for Mitigating Structural Biases in Natural Language Inference

A Generative Approach for Mitigating Structural Biases in Natural Language Inference

Paper · 2022 · Dimion Asael, Zachary Ziegler and Yonatan Belinkov

Summary

NLI benchmarks such as SNLI and MNLI allow classifiers to score well while using only part of the input, such as the hypothesis by itself or the words shared by premise and hypothesis. The authors call these structural biases: a chosen subset B of the input that should carry no label signal but does. Instead of predicting the label directly, they train encoder–decoder models (BERT-, BART- and T5-based) to produce the remaining input R from B and the label, then classify with Bayes' rule. Setting the label prior given B to uniform makes it provably impossible for the model to predict from B alone. The resulting model, GEN, was tested on synthetically injected bias and on natural hypothesis-only and overlap biases. It scores about the same on standard and hard test sets and correlates only weakly with a biased model. Its accuracy, however, falls below discriminative baselines. A discriminative fine-tuning stage (GEN-FT) recovers much of that loss while letting a little bias back in. The approach also works with T5 up to 11B parameters, including with prompt-tuning.

Key points

  • Bias is measured two ways: the o.o.d generalization gap ∆ (regular test vs. a "hard" subset where a hypothesis-only model fails) and the correlation ρ with a fully biased model p(y | B).
  • On SNLI, earlier debiasing methods still have gaps of roughly +3 to +14 points. GEN (BART) reaches 70.58 on test and 72.19 on hard test (∆ −1.61). On MNLI it gets 64.09 vs. 65.74 (∆ −1.65).
  • Overlap bias on MNLI mismatched dev: the discriminative model gets 86.44/79.72 (∆ 6.73, ρ 0.171), while GEN-FT gets 79.87/74.98 (∆ 4.89, ρ 0.106).
  • GEN-FT pipeline: (1) train and freeze a prior p(y | B); (2) train the generator p(R | y, B); (3) fine-tune with label cross-entropy through Bayes' rule using the frozen prior; (4) at test time, swap in a uniform prior.
  • With T5-XXL (11B) and prompt-tuning, the discriminative model gets 92.88/85.46 (∆ 7.42), GEN gets 79.91/79.70 (∆ 0.21), and GEN-FT gets 89.68/85.22 (∆ 4.46). Code is at technion-cs-nlp/Generative-NLI.

Interesting ideas

  • I1. A bias is defined by which part of the input should be uninformative, not by a known surface cue like negation words, so you can test for it without knowing its exact form.
  • I2. The "hard" split is built from items that a deliberately crippled model gets wrong, and agreement with that crippled model is reported alongside accuracy.
  • I3. A generative classifier scores each label by how well it explains the rest of the input; a flat prior over labels means the biased part cannot vote on its own.
  • I4. During fine-tuning, a frozen bias model takes on the spurious signal, and it is thrown away at inference so the main component stays cleaner.
  • I5. Injecting an artificial cue into a controlled fraction of the training data shows how fast a model's robustness degrades as bias grows.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): as a negative control, run a reader that sees only the question (or only the overlap with the text); any accuracy above chance shows our test set leaks answers.
  • I2 (answer): for school tests and fable QA, keep a hard subset where a question-only baseline fails, and report the gap between it and the full set as a shortcut detector.
  • I3 (frames): when choosing between candidate frames or speech-act readings, prefer the one whose roles best account for the remaining tokens of the sentence, rather than the one cued by a single trigger word.
  • I4 (learn): when training on silver data with a known artefact (e.g., a witness parser's habit), model that artefact separately while training, then drop it when the student runs.
  • I5 (tree): add a spurious marker correlated with a relation label to a growing share of treebank sentences, and check whether the perceptron parser or gparse starts relying on it in LAS on clean data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Harvard UniversityCambridge · United States112 works · 42,023 citations🎓Technion – Israel Institute of TechnologyHaifa · Israel49 works · 5,306 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,426 works

Cite

Cite as Dimion Asael, Zachary Ziegler and Yonatan Belinkov. 2022. A Generative Approach for Mitigating Structural Biases in Natural Language Inference. Proceedings of the 11th Joint Conference on Lexical and Computational Semantics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2022.starsem-1.16

@inproceedings{asael2022generative,
  title = {{A Generative Approach for Mitigating Structural Biases in Natural Language Inference}},
  author = {Dimion Asael, Zachary Ziegler and Yonatan Belinkov},
  booktitle = {Proceedings of the 11th Joint Conference on Lexical and Computational Semantics},
  year = {2022},
  doi = {10.18653/v1/2022.starsem-1.16},
  url = {https://doi.org/10.18653/v1/2022.starsem-1.16}
}

References

33 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aishwarya Agrawal, Dhruv Batra, Devi Parikh, and Aniruddha Kembhavi. 2018. Don’t just assume; look and answer: Overcoming priors for visual question answering. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 4971– 4980. IEEE Computer Society. find
  2. Yonatan Belinkov, Adam Poliak, Stuart Shieber, Benjamin Van Durme, and Alexander Rush. 2019a. Don’t take the premise for granted: Mitigating artifacts in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 877–891, Florence, Italy. Association for Computational Linguistics.
  3. Yonatan Belinkov, Adam Poliak, Stuart Shieber, Benjamin Van Durme, and Alexander Rush. 2019b. On adversarial removal of hypothesis-only bias in natural language inference. In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019), pages 256–262, Minneapolis, Minnesota. Association for Computational Linguistics.
  4. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642, Lisbon, Portugal. Association for Computational Linguistics. ↗
  5. Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. e-SNLI: Natural language inference with natural language explanations. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada, pages 9560– 9572. ↗
  6. Anubrata Das, Samreen Anjum, and Danna Gurari. 2019. Dataset bias: A case study for visual question answering. Proceedings of the Association for Information Science and Technology, 56(1):58–67. ↗
  7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. ↗
  8. Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. 2018. Annotation artifacts in natural language inference data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 107–112, New Orleans, Louisiana. Association for Computational Linguistics. ↗
  9. He He, Sheng Zha, and Haohan Wang. 2019. Unlearn dataset bias in natural language inference by fitting the residual. In Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019), pages 132–142, Hong Kong, China. Association for Computational Linguistics.
  10. Rabeeh Karimi Mahabadi, Yonatan Belinkov, and James Henderson. 2020. End-to-end bias mitigation by modelling biases in corpora. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8706–8716, Online. Association for Computational Linguistics. ↗
  11. Divyansh Kaushik and Zachary C. Lipton. 2018. How much reading does reading comprehension require? a critical investigation of popular benchmarks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 5010– 5015, Brussels, Belgium. Association for Computational Linguistics.
  12. Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos Araya, Siqi Yan, et al. 2020. Captum: A unified and generic model interpretability library for pytorch. arXiv preprint arXiv:2009.07896. ↗
  13. Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691. ↗
  14. Mike Lewis and Angela Fan. 2019. Generative question answering: Learning to answer the whole question. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. find
  15. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871–7880, Online. Association for Computational Linguistics. ↗
  16. Varun Manjunatha, Nirat Saini, and Larry S. Davis. 2019. Explicit bias discovery in visual question answering models. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019, pages 9562– 9571. Computer Vision Foundation / IEEE. find
  17. Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3428–3448, Florence, Italy. Association for Computational Linguistics. ↗
  18. Michael Mendelson and Yonatan Belinkov. 2021. Debiasing methods in natural language understanding make bias more accessible. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1545–1557, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. ↗
  19. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics. ↗
  20. Adam Poliak, Jason Naradowsky, Aparajita Haldar, Rachel Rudinger, and Benjamin Van Durme. 2018. Hypothesis only baselines in natural language inference. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, pages 180–191, New Orleans, Louisiana. Association for Computational Linguistics. ↗
  21. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21:1– 67. ↗
  22. Sascha Rothe, Shashi Narayan, and Aliaksei Severyn. 2020. Leveraging pre-trained checkpoints for sequence generation tasks. Transactions of the Association for Computational Linguistics, 8:264–280. ↗
  23. Victor Sanh, Thomas Wolf, Yonatan Belinkov, and Alexander M Rush. 2021. Learning from others’ mistakes: Avoiding dataset biases without modeling them. In International Conference on Learning Representations. ↗
  24. Tal Schuster, Darsh Shah, Yun Jie Serene Yeo, Daniel Roberto Filizzola Ortiz, Enrico Santus, and Regina Barzilay. 2019. Towards debiasing fact verification models. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3419–3425, Hong Kong, China. Association for Computational Linguistics.
  25. August 2017, volume 70 of Proceedings of Machine Learning Research, pages 3319–3328. PMLR. find
  26. Masatoshi Tsuchiya. 2018. Performance impact caused by hidden bias of training data for recognizing textual entailment. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA).
  27. Prasetya Ajie Utama, Nafise Sadat Moosavi, and Iryna Gurevych. 2020a. Mind the trade-off: Debiasing NLU models without degrading the in-distribution performance. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8717–8729, Online. Association for Computational Linguistics.
  28. Prasetya Ajie Utama, Nafise Sadat Moosavi, and Iryna Gurevych. 2020b. Towards debiasing NLU models from unknown biases. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7597–7610, Online. Association for Computational Linguistics. find
  29. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1112–1122, New Orleans, Louisiana. Association for Computational Linguistics. ↗
  30. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2019. Huggingface’s transformers: State-ofthe-art natural language processing. ArXiv, pages arXiv–1910. find
  31. Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. 2018. Texygen: A benchmarking platform for text generation models. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, Ann Arbor, MI, USA, July 08-12, 2018, pages 1097–1100. ACM. ↗
  32. Joe Stacey, Pasquale Minervini, Haim Dubossarsky, Sebastian Riedel, and Tim Rocktäschel. 2020. Avoiding the Hypothesis-Only Bias in Natural Language Inference via Ensemble Adversarial Training. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8281–8291, Online. Association for Computational Linguistics.
  33. Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/asael-2022-generative-approach-mitigating-structural-biases/