Elamur Research⌕
☰
home / papers / 2018 / Performance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailment

Performance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailment

Summary

This paper asks whether a large RTE corpus contains systematic artifacts that let a model guess the entailment label without doing real inference. The author proposes a quality check modelled on a statistical hypothesis test. Since an entailment label should, by definition, depend on the premise, the "null hypothesis" states that the label can be determined from the hypothesis sentence alone. This is tested by comparing a multinomial Naive Bayes classifier that sees only hypothesis-sentence unigrams with a majority-class baseline, using a sign test.

The check was applied to SNLI and SICK. On SICK, the hypothesis-only model is no better than the baseline. On SNLI it is far better, which reveals a hidden bias. The SNLI test set is then split into an "easy" part (pairs the hypothesis-only model gets right) and a "hard" part. Two LSTM-based RTE models show a large performance drop on the hard part. This suggests that much of their apparent skill comes from exploiting the bias rather than from reasoning over sentence pairs.

Key points

  • Hypothesis-only model: multinomial NB (Wang and Manning, 2012) with word unigrams of the hypothesis as features, implemented with scikit-learn. It is compared to a baseline that always outputs the most frequent label.
  • SNLI result: 63.3% accuracy versus 34.3% for the baseline. The sign test gives p = 5.7e−202, and the confusion matrix shows real per-sentence predictions across all three labels.
  • SICK result: both models reach 56.7% (p = 0.65). The NB model essentially outputs "neutral" for nearly every item, so no exploitable bias is detected.
  • SNLI test split:
    • empirical easy set E_e: 6,219 pairs (63.3%);
    • empirical hard set H_e: 3,605 pairs (36.7%);
    • labels are spread fairly evenly across entailment, neutral and contradiction in both subsets.
  • Models probed:
    • a "parallel" LSTM model: 300-d GloVe embeddings, two 100-d sentence LSTMs concatenated, three tanh layers and softmax;
    • an attention-less "sequential" LSTM model following Rocktäschel et al. (2015).
    • Both perform much worse on H_e than on E_e.

Interesting ideas

  • I1. Remove the input that should be essential. Deliberately withholding a piece of the input that the task definition requires makes a cheap probe for annotation artifacts. If a classifier still beats chance, the data leaks label information. The same probe could be applied to QA (answers without the question), parallel-corpus filtering, or any paired-input dataset.
  • I2. Use a deliberately weak probe. A bag-of-words Naive Bayes model is a strong choice for an artifact detector precisely because it is weak. Any success it has can only come from surface lexical cues, not from genuine understanding.
  • I3. Frame the audit as a significance test. Comparing the probe against a majority baseline with a sign test, rather than just reporting accuracy, gives a principled yes/no verdict. The test needs only the data itself, with no multiple annotations or agreement scores required.
  • I4. Inspect the confusion matrix, not just accuracy. On SICK, identical accuracies hide a degenerate always-"neutral" model. Checking the matrix distinguishes true predictive signal from majority-label behaviour.
  • I5. Build easy/hard splits from the probe. Splitting a test set by whether the artifact model succeeds yields a hard subset for stress-testing stronger models. This is reusable for evaluating any new benchmark or model, including for low-resource languages such as Ukrainian.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

95 papers of our library cite it (of the 140673 whose bibliographies we read); 11 of them have a page here, 84 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{tsuchiya2018performance,
  title = {{Performance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailment}},
  author = {Tsuchiya, Masatoshi},
  booktitle = {Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)},
  year = {2018},
  doi = {10.63317/26mjmiipm2qf},
  url = {https://doi.org/10.63317/26mjmiipm2qf}
}

References

30 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Al Khatib, K., Wachsmuth, H., Kiesel, J., Hagen, M., and Stein, B. (2016). A news editorial corpus for mining argumentation strategies. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 3433–3443, Osaka, Japan, December. find
  2. The COLING 2016 Organizing Committee. find
  3. Artstein, R. and Poesio, M. (2008). Inter-coder agreement for computational linguistics. Computational Linguistics, 34(4):555–596. ↗
  4. Bos, J. and Markert, K. (2005). Recognising textual entailment with logical inference. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 628–635, Vancouver, British Columbia, Canada, October. Association for Computational Linguistics. find our card
  5. Bowman, S. R., Angeli, G., Potts, C., and Manning, C. D. (2015). A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642, Lisbon, Portugal, September. Association for Computational Linguistics. ↗
  6. Cheng, J., Dong, L., and Lapata, M. (2016). Long short-term memory-networks for machine reading. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 551–561, Austin, Texas, November. Association for Computational Linguistics. find our card
  7. Condoravdi, C., Crouch, D., de Paiva, V., Stolle, R., and Bobrow, D. G. (2003). Entailment, intensionality and text understanding. In Graeme Hirst et al., editors, Proceedings of the HLT-NAACL 2003 Workshop on Text Meaning, pages 38–45. find
  8. Craggs, R. and Wood, M. M. (2005). Evaluating discourse and dialogue coding schemes. Computational Linguistics, 31(3):289–296, September. ↗
  9. Liu, P., Qiu, X., Chen, J., and Huang, X. (2016a). Deep fusion lstms for text semantic matching. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1034–1043, Berlin, Germany, August. Association for Computational Linguistics. find
  10. Liu, P., Qiu, X., Zhou, Y., Chen, J., and Huang, X. (2016b). Modelling interaction of sentence pair with coupled-lstms. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1703–1712, Austin, Texas, November. Association for Computational Linguistics. find
  11. MacCartney, B. and Manning, C. D. (2009). An extended model of natural logic. In Proceedings of the Eight International Conference on Computational Semantics, pages 140–156, Tilburg, The Netherlands, January. Association for Computational Linguistics. find
  12. Marelli, M., Bentivogli, L., Baroni, M., Bernardi, R., Menini, S., and Zamparelli, R. (2014a). Semeval-2014 task 1: Evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 1–8, Dublin, Ireland, August. Association for Computational Linguistics and Dublin City University. find our card
  13. Marelli, M., Menini, S., Baroni, M., Bentivogli, L., Bernardi, R., and Zamparelli, R. (2014b). A SICK cure for the evaluation of compositional distributional semantic models. In Nicoletta Calzolari (Conference Chair), et al., editors, Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 216–223, Reykjavik, Iceland, may. European Language Resources Association (ELRA). find
  14. Mathet, Y., Widlöcher, A., Fort, K., François, C., Galibert, O., Grouin, C., Kahn, J., Rosset, S., and Zweigenbaum, P. (2012). Manual corpus annotation: Giving meaning to the evaluation metrics. In Proceedings of COLING 2012: Posters, pages 809–818, Mumbai, India, December. find
  15. The COLING 2012 Organizing Committee. find
  16. Mou, L., Men, R., Li, G., Xu, Y., Zhang, L., Yan, R., and Jin, Z. (2016a). Natural language inference by tree-based convolution and heuristic matching. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 130–136, Berlin, Germany, August. Association for Computational Linguistics. ↗ our card
  17. Mou, L., Meng, Z., Yan, R., Li, G., Xu, Y., Zhang, L., and Jin, Z. (2016b). How transferable are neural networks in nlp applications? In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 479–489, Austin, Texas, November. Association for Computational Linguistics. ↗
  18. Parikh, A., Täckström, O., Das, D., and Uszkoreit, J. (2016). A decomposable attention model for natural language inference. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2249–2255, Austin, Texas, November. Association for Computational Linguistics. ↗ our card
  19. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830. ↗
  20. Pennington, J., Socher, R., and Manning, C. (2014). Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532– 1543, Doha, Qatar, October. Association for Computational Linguistics. find
  21. Reidsma, D. and Carletta, J. (2008). Reliability measurement without limits. Computational Linguistics, 34(3):319–326, September. ↗
  22. Rocktäschel, T., Grefenstette, E., Hermann, K. M., Kociský, T., and Blunsom, P. (2015). Reasoning about entailment with neural attention. In Proceedings of The International Conference on Learning Representations (ICLR2016). ↗ our card
  23. Sabou, M., Bontcheva, K., Derczynski, L., and Scharl, A. (2014). Corpus annotation through crowdsourcing: Towards best practice guidelines. pages 859–866. find
  24. Sha, L., Chang, B., Sui, Z., and Li, S. (2016). Reading and thinking: Re-read lstm unit for textual entailment recognition. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2870–2879, Osaka, Japan, December. find
  25. The COLING 2016 Organizing Committee. find
  26. Wang, S. and Jiang, J. (2016). Learning natural language inference with lstm. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1442–1451, San Diego, California, June. Association for Computational Linguistics. find
  27. Wang, S. and Manning, C. (2012). Baselines and bigrams: Simple, good sentiment and topic classification. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 90–94, Jeju Island, Korea, July. Association for Computational Linguistics. find
  28. Yin, W., Schà 14 tze, H., Xiang, B., and Zhou, B. (2016). Abcnn: Attention-based convolutional neural network for modeling sentence pairs. Transactions of the Association for Computational Linguistics, 4:259–272. ↗ our card
  29. Young, P., Lai, A., Hodosh, M., and Hockenmaier, J. (2014). From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics, 2:67–78. ↗
  30. Zhang, C., Bengio, S., Hardt, M., Recht, B., and Vinyals, O. (2017). Understanding deep learning requires rethinking generalization. In Proceedings of The International Conference on Learning Representations (ICLR2016). ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/tsuchiya-2018-performance-impact-caused-hidden-bias/