Elamur Research
☰
home / papers / 2019 / Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual

Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/D19-6115.pdf

Summary

NLI models trained on crowdsourced corpora often latch onto shallow cues that correlate with labels only in the training data. Examples include negation words signalling contradiction and high premise–hypothesis word overlap signalling entailment. Such models fail on challenge sets where these correlations are absent. The authors frame dataset bias as label shift: the biased features still occur at test time, but their relationship to the label changes. They propose DRiFt (Debias by Residual Fitting), which works in two steps:

  1. Train a deliberately limited "biased" model on features known to carry bias.
  2. Train the main model on the residual of that biased model, so learning focuses on examples the shortcut cannot solve.

At inference, only the debiased model is used. DRiFt was applied to three strong NLI architectures trained on SNLI and MNLI. On challenge data such as HANS and the Naik et al. stress tests, the debiased models outperform maximum-likelihood baselines, with only modest losses on the original test sets. The approach matters because it corrects for known biases without stripping potentially useful features from the input.

Key points

  • Bias is defined formally through a split of the input into stable "semantic" features g(x) and biased features b(x). Under this split, p(y | b(x)) differs between training and test distributions.
  • DRiFt variants use different bias-only models, such as hypothesis-only, word-overlap and CBOW-based classifiers (e.g., DRiFt-CBOW). The debiased models are DA, ESIM and BERT.
  • Evaluation covers HANS (word-overlap heuristic) and the STRESS sets of Naik et al. (2018), focusing on the word overlap and negation subsets. It also includes a reweighting/filtering baseline (RM).
  • BERT keeps in-distribution performance nearly intact whichever biased model is used. DA, which is essentially a bag-of-words model, degrades most, especially with DRiFt-CBOW.
  • The method is concurrent with and similar to Clark et al. (2019). It depends on prior knowledge of the bias, and the authors highlight more general debiasing methods as future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

97 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 95 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d19-6115

@inproceedings{he2019unlearn,
  title = {{Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual}},
  author = {He, He and Zha, Sheng and Wang, Haohan},
  booktitle = {Proceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019)},
  year = {2019},
  doi = {10.18653/v1/d19-6115},
  url = {https://doi.org/10.18653/v1/d19-6115}
}

References

44 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. W. Hu, G. Niu, I. Sato, and M. Sugiyama. 2018. Does distributionally robust supervised learning give robust classifiers? In International Conference on Machine Learning (ICML). find
  2. A. Agrawal, D. Batra, and D. Parikh. 2016. Analyzing the behavior of visual question answering models. In Empirical Methods in Natural Language Processing (EMNLP). find
  3. R. Jia and P. Liang. 2017. Adversarial examples for evaluating reading comprehension systems. In Empirical Methods in Natural Language Processing (EMNLP). find our card
  4. J. Andreas. 2019. Good-enough compositional data augmentation. arXiv. find our card
  5. Y. Belinkov, A. Poliak, S. M. Shieber, B. V. Durme, and A. M. Rush. 2019. Don’t take the premise for granted: Mitigating artifacts in natural language inference. In Association for Computational Linguistics (ACL). V. find our card
  6. Karpukhin, O. Levy, J. Eisenstein, and M. Ghazvininejad. 2019. Training on synthetic noise improves robustness to natural noise in machine translation. arXiv. find
  7. D. Kaushik and Z. C. Lipton. 2018. How much reading does reading comprehension require? a critical investigation of popular benchmarks. In Empirical Methods in Natural Language Processing (EMNLP). find our card
  8. S. Ben-David, J. Blitzer, K. Crammer, and F. Pereira. 2006. Analysis of representations for domain adaptation. In Advances in Neural Information Processing Systems (NeurIPS), pages 137–144. find
  9. T. Bolukbasi, K. Chang, J. Y. Zou, V. Saligrama, and A. T. Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in Neural Information Processing Systems (NeurIPS), pages 4349–4357. find our card
  10. D. Kingma and J. Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. ↗
  11. Z. C. Lipton, Y. Wang, and A. J. Smola. 2018. Detecting and correcting for label shift with black box predictors. In International Conference on Machine Learning (ICML). find
  12. S. Bowman, G. Angeli, C. Potts, and C. D. Manning. 2015. A large annotated corpus for learning natural language inference. In Empirical Methods in Natural Language Processing (EMNLP). find
  13. N. F. Liu, R. Schwartz, and N. A. Smith. 2019. Inoculation by fine-tuning: A method for analyzing challenge datasets. In North American Association for Computational Linguistics (NAACL). find
  14. Q. Chen, X. Zhu, Z. Ling, S. Wei, H. Jiang, and D. Inkpen. 2017. Enhanced LSTM for natural language inference. In Association for Computational Linguistics (ACL). find
  15. R. T. McCoy, E. Pavlick, and T. Linzen. 2019. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. arXiv preprint arXiv:1902.01007. ↗ our card
  16. C. Clark, M. Yatskar, and L. Zettlemoyer. 2019. Don’t take the easy way out: Ensemble based methods for avoiding known dataset biases. In Empirical Methods in Natural Language Processing (EMNLP). find
  17. L. Mou, R. Men, G. Li, Y. Xu, L. Zhang, R. Yan, and Z. Jin. 2016. Natural language inference by tree-based convolution and heuristic matching. In Association for Computational Linguistics (ACL). find our card
  18. J. Devlin, M. Chang, K. Lee, and K. Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In North American Association for Computational Linguistics (NAACL). find
  19. A. Naik, A. Ravichander, N. Sadeh, C. Rose, and G. Neubig. 2018. Stress test evaluation for natural language inference. In International Conference on Computational Linguistics (COLING). find
  20. J. Duchi and H. Namkoong. 2018. Learning models with uniform performance via distributionally robust optimization. arXiv preprint arXiv:1810.08750. ↗
  21. M. Glockner, V. Shwartz, and Y. Goldberg. 2018. Breaking NLI systems with sentences that require simple lexical inferences. In Association for Computational Linguistics (ACL). find our card
  22. A. Parikh, O. Täckström, D. Das, and J. Uszkoreit. 2016. A decomposable attention model for natural language inference. In Empirical Methods in Natural Language Processing (EMNLP). find our card
  23. H. Gonen and Y. Goldberg. 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. arXiv preprint arXiv:1903.03862. ↗ our card
  24. J. Pennington, R. Socher, and C. D. Manning. 2014. GloVe: Global vectors for word representation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543. find
  25. S. Gururangan, S. Swayamdipta, O. Levy, R. Schwartz, S. R. Bowman, and N. A. Smith. 2018. Annotation artifacts in natural language inference data. In North American Association for Computational Linguistics (NAACL). find
  26. A. Poliak, J. Naradowsky, A. Haldar, R. Rudinger, and B. V. Durme. 2018. Hypothesis only baselines in natural language inference. arXiv preprint arXiv:1805.01042. ↗
  27. M. T. Ribeiro, S. Singh, and C. Guestrin. 2018. Semantically equivalent adversarial rules for debugging NLP models. In Association for Computational Linguistics (ACL). find
  28. K. Zhang, B. Schlkopf, K. Muandet, and Z. Wang. 2013. Domain adaptation under target and conditional shift. In International Conference on Machine Learning (ICML). find
  29. K. Sakaguchi, R. L. Bras, C. Bhagavatula, and Y. Choi. 2019. WINOGRANDE: An adversarial winograd schema challenge at scale. arXiv preprint arXiv:1907.10641. ↗
  30. Y. Zhang, J. Baldridge, and L. He. 2019c. PAWS: Paraphrase adversaries from word scrambling. In North American Association for Computational Linguistics (NAACL). find our card
  31. B. Scholkopf, D. Janzing, J. Peters, E. Sgouritsa, K. Zhang, and J. Mooij. 2012. On causal and anti-causal learning. In International Conference on Machine Learning (ICML). find
  32. J. Zhao, T. Wang, M. Yatskar, R. Cotterell, V. Ordonez, and K. Chang. 2019. Gender bias in contextualized word embeddings. In North American Association for Computational Linguistics (NAACL). find our card
  33. R. Schwartz, M. Sap, Y. Konstas, L. Zilles, Y. Choi, and N. A. Smith. 2017. The effect of different writing tasks on linguistic style: A case study of the ROC story cloze task. In Computational Natural Language Learning (CoNLL). find
  34. J. Zhao, T. Wang, M. Yatskar, V. Ordonez, and K. Chang. 2018a. Gender bias in coreference resolution:evaluation and debiasing methods. In North American Association for Computational Linguistics (NAACL). find our card
  35. H. Shimodaira. 2000. Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of Statistical Planning and Inference, 90:227–244. find
  36. J. Zhao, Y. Zhou, Z. Li, W. Wang, and K. Chang. 2018b. Learning gender-neutral word embeddings. In Empirical Methods in Natural Language Processing (EMNLP). find our card
  37. A. Torralba and A. Efros. 2011. Unbiased look at dataset bias. In Computer Vision and Pattern Recognition (CVPR). find
  38. D. Tsipras, S. Santurkar, L. Engstrom, A. Turner, and A. Madry. 2019. Robustness may be at odds with accuracy. In International Conference on Learning Representations (ICLR). find
  39. H. Wang, Z. He, Z. C. Lipton, and E. P. Xing. 2019a. Learning robust representations by projecting superficial statistics out. In International Conference on Learning Representations (ICLR). find
  40. T. Wang, J. Zhao, M. Yatskar, K. Chang, and V. Ordonez. 2019b. Balanced datasets are not enough: Estimating and mitigating gender bias in deep image representations. In International Conference on Computer Vision (ICCV). find
  41. A. Williams, N. Nangia, and S. R. Bowman. 2017. A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426. ↗
  42. R. Zellers, A. Holtzman, Y. Bisk, A. Farhadi, and Y. Choi. 2019. HellaSwag: Can a machine really finish your sentence? In Association for Computational Linguistics (ACL). find
  43. G. Zhang, B. Bai, J. Liang, K. Bai, S. Chang, M. Yu, C. Zhu, and T. Zhao. 2019a. Selection bias explorations and debias methods for natural language sentence matching datasets. In Association for Computational Linguistics (ACL). find
  44. H. Zhang, Y. Yu, J. Jiao, E. P. Xing, L. E. Ghaoui, and M. I. Jordan. 2019b. Theoretically principled trade-off between robustness and accuracy. arXiv preprint arXiv:1901.08573. ↗