Elamur Research
☰
home / papers / 2019 / Don't Take the Premise for Granted: Mitigating Artifacts in Natural Language Inference

Don't Take the Premise for Granted: Mitigating Artifacts in Natural Language Inference

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/P19-1084.pdf

Summary

Many NLI corpora contain hypothesis-only biases: cues in the hypothesis, such as negation words linked to contradiction, that let a classifier score well without examining the premise. Models that exploit these cues tend to fail on data with other artifacts or none. The authors reframe the objective as modeling the premise given the hypothesis and label. This target cannot be solved from the hypothesis alone. Because it is intractable to estimate directly, they derive two approximations that keep a standard NLI classifier but add an adversarial term. The first trains a hypothesis-only classifier on a shared hypothesis encoder and reverses its gradients into the encoder. The second pairs hypotheses with randomly swapped premises and penalizes the model for still predicting the original label. On a controlled synthetic task, both methods recover the true premise–hypothesis rule where the baseline is misled. When trained on SNLI and evaluated elsewhere, they transfer better than the baseline on 9 of 12 target datasets. This offers a training-time remedy that does not require building new bias-free corpora.

Key points

  • Methods:
    • Method 1 (hypothesis-only classifier) uses a gradient reversal layer, as in domain-adversarial networks, but needs no domain label.
    • Method 2 (negative sampling) blocks gradients to the premise encoder for the sampled premises.
    • Both are controlled by hyper-parameters α and β.
  • Synthetic test: The synthetic data (about 1K training and 1K test examples) has an artifact "c" attached to every entailed hypothesis. The baseline latches onto this artifact, while both proposed methods learn the correct letter-matching rule.
  • Transfer results: Gains are largest when target datasets have biases that differ from SNLI or have no noticeable biases. Pushing harder against biases improves transfer but lowers accuracy on the source dataset.
  • Fine-tuning: Models pre-trained on SNLI were fine-tuned on MNLI and SICK (SICK has about 10K examples). The proposed methods gave little benefit on MNLI, likely because its biases resemble SNLI's, but helped on SICK.
  • Code: Available at github.com/azpoliak/robust-nli.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Natural language processing

Cited by

49 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 46 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p19-1084

@inproceedings{belinkov2019don,
  title = {{Don't Take the Premise for Granted: Mitigating Artifacts in Natural Language Inference}},
  author = {Belinkov, Yonatan and Poliak, Adam and Shieber, Stuart and Durme, Benjamin Van and Rush, Alexander},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1084},
  url = {https://doi.org/10.18653/v1/p19-1084}
}

References

69 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. URL http://www.aclweb.org/anthology/ P17-2097. Martı́n Abadi and David G. Andersen. Learning to protect communications with adversarial neural cryptography. arXiv, 2016. URL https:// arxiv.org/abs/1610.06918. ↗
  2. Qian Chen, Xiaodan Zhu, Zhen-Hua Ling, Si Wei, Hui Jiang, and Diana Inkpen. Enhanced LSTM for Natural Language Inference. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1657–1668. Association for Computational Linguistics, 2017. doi: 10.18653/v1/ P17-1152. URL http://www.aclweb.org/ anthology/P17-1152. ↗
  3. Aishwarya Agrawal, Dhruv Batra, Devi Parikh, and Aniruddha Kembhavi. Don’t Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering. arXiv preprint arXiv:1712.00377, 2017. ↗
  4. Yonatan Belinkov and Yonatan Bisk. Synthetic and Natural Noise Both Break Neural Machine Translation. In International Conference on Learning Representations, 2018. URL https:// openreview.net/forum?id=BJ8vJebC-. ↗
  5. Jianpeng Cheng, Li Dong, and Mirella Lapata. Long Short-Term Memory-Networks for Machine Reading. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 551–561, Austin, Texas, November 2016. Association for Computational Linguistics. URL https://aclweb.org/ anthology/D16-1053. ↗ our card
  6. Yonatan Belinkov and James Glass. Analysis Methods in Neural Language Processing: A Survey. Transactions of the Association for Computational Linguistics (TACL), 7:49–72, 2019. doi: 10.1162/tacl\ a\ 00254. URL https://doi.org/10.1162/ tacl a 00254. ↗ our card
  7. Alexis Conneau, Douwe Kiela, Holger Schwenk, Loı̈c Barrault, and Antoine Bordes. Supervised Learning of Universal Sentence Representations from Natural Language Inference Data. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 670–680. Association for Computational Linguistics, 2017. URL http://aclweb.org/anthology/D17-1070. ↗ our card
  8. Yonatan Belinkov, Adam Poliak, Stuart M. Shieber, Benjamin Van Durme, and Alexander Rush. On Adversarial Removal of Hypothesis-only Bias in Natural Language Inference. In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM, Oral presentation), June 2019. find
  9. Robin Cooper, Dick Crouch, Jan Van Eijck, Chris Fox, Johan Van Genabith, Jan Jaspars, Hans Kamp, David Milward, Manfred Pinkal, Massimo Poesio, and Steven Pullman. Using the framework. Technical Report LRE 62-051 D-16, The FraCaS Consortium, 1996. find
  10. Yoshua Bengio and Jean-Sébastien Senecal. Quick Training of Probabilistic Neural Nets by Importance Sampling. In Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics, AISTATS 2003, Key West, Florida, USA, January 3-6, 2003, pp. 1–9, 2003. URL http://research.microsoft.com/en-us/ um/cambridge/events/aistats2003/ proceedings/164.pdf. ↗
  11. Ido Dagan, Oren Glickman, and Bernardo Magnini. The PASCAL recognising textual entailment challenge. In Machine learning challenges. evaluating predictive uncertainty, visual object classification, and recognising tectual entailment, pp. 177– 190. Springer, 2006. find
  12. Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. In Advances in Neural Information Processing Systems, pp. 4349–4357, 2016. find our card
  13. Chaoqun Duan, Lei Cui, Xinchi Chen, Furu Wei, Conghui Zhu, and Tiejun Zhao. Attention-Fused Deep Matching Network for Natural Language Inference. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, pp. 4033–4040. International Joint Conferences on Artificial Intelligence Organization, 7 2018. doi: 10.24963/ijcai.2018/561. URL https://doi.org/10.24963/ijcai.2018/561. ↗
  14. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 2015. find
  15. Javid Ebrahimi, Daniel Lowd, and Dejing Dou. On Adversarial Examples for Character-Level Neural Machine Translation. In Proceedings of the 27th International Conference on Computational Linguistics, pp. 653–663. Association for Computational Linguistics, 2018. URL http://aclweb.org/ anthology/C18-1055. ↗
  16. Zheng Cai, Lifu Tu, and Kevin Gimpel. Pay Attention to the Ending: Strong Neural Baselines for the ROC Story Cloze Task. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 616–622. Association for Computational Linguistics, 2017. doi: 10.18653/v1/P17-2097. ↗
  17. Yanai Elazar and Yoav Goldberg. Adversarial Removal of Demographic Attributes from Text Data. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 11–21. Association for Computational Linguistics, 2018. URL http://aclweb.org/ anthology/D18-1002. ↗ our card
  18. Conference of The Association for Machine Translation in the Americas (Volume 1: Research Track, pp. 68–79, March 2018. find
  19. Robin Jia and Percy Liang. Adversarial Examples for Evaluating Reading Comprehension Systems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2011–2021, Copenhagen, Denmark, September 2017. URL https://www.aclweb.org/ anthology/D17-1215. ↗ our card
  20. Yaroslav Ganin and Victor Lempitsky. Unsupervised Domain Adaptation by Backpropagation. In International Conference on Machine Learning, pp. 1180–1189, 2015. find
  21. Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. Domain-adversarial training of neural networks. The Journal of Machine Learning Research, 17(1): 2096–2030, 2016. find
  22. K. Kafle and C. Kanan. Answer-Type Prediction for Visual Question Answering. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4976–4984, June 2016. doi: 10.1109/ CVPR.2016.538. find
  23. Max Glockner, Vered Shwartz, and Yoav Goldberg. Breaking NLI Systems with Sentences that Require Simple Lexical Inferences. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 650–655, Melbourne, Australia, July 2018. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/P18-2103. ↗ our card
  24. Kushal Kafle and Christopher Kanan. Visual question answering: Datasets, algorithms, and future challenges. Computer Vision and Image Understanding, 163:3–20, 2017. find
  25. Dongyeop Kang, Tushar Khot, Ashish Sabharwal, and Eduard Hovy. AdvEntuRe: Adversarial Training for Textual Entailment with Knowledge-Guided Examples. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2418–2428, Melbourne, Australia, July 2018. Association for Computational Linguistics. URL http://www.aclweb.org/ anthology/P18-1225. ↗
  26. Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and Harnessing Adversarial Examples. In International Conference on Learning Representations (ICLR), 2015. find our card
  27. Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the V in VQA matter: Elevating the role of image understanding in Visual Question Answering. In CVPR, volume 1, pp. 3, 2017. find our card
  28. Divyansh Kaushik and Zachary C. Lipton. How Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 5010–5015, Brussels, Belgium, October-November 2018. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/ D18-1546. ↗ our card
  29. Gabriel Grand and Yonatan Belinkov. Adversarial Regularization for Visual Question Answering: Strengths, Shortcomings, and Side Effects, June 2019. find
  30. Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. Annotation Artifacts in Natural Language Inference Data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pp. 107–112, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/N18-2017. ↗
  31. Tushar Khot, Ashish Sabharwal, and Peter Clark. SciTail: A Textual Entailment Dataset from Science Question Answering. In AAAI, 2018. find
  32. Alice Lai, Yonatan Bisk, and Julia Hockenmaier. Natural Language Inference from Multiple Premises. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 100–109, Taipei, Taiwan, November 2017. Asian Federation of Natural Language Processing. URL http://www.aclweb.org/ anthology/I17-1011. ↗
  33. Michael Gutmann and Aapo Hyvärinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 297–304, 2010. find
  34. Yitong Li, Timothy Baldwin, and Trevor Cohn. Towards Robust and Privacy-preserving Text Representations. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 25–30. Association for Computational Linguistics, 2018. URL http://aclweb.org/anthology/P18-2005. ↗
  35. Georg Heigold, Günter Neumann, and Josef van Genabith. How Robust Are Character-Based Word Embeddings in Tagging and MT Against Wrod Scramlbing or Randdm Nouse? In Proceedings of the 13th Yang Liu, Chengjie Sun, Lei Lin, and Xiaolong Wang. Learning natural language inference using bidirectional LSTM model and inner-attention. arXiv preprint arXiv:1605.09090, 2016. 2016. doi: 10.18653/v1/D16-1244. URL http://www.aclweb.org/anthology/D16-1244. ↗ our card
  36. Ellie Pavlick and Chris Callison-Burch. Most “babies” are “little” and most “problems” are “huge”: Compositional entailment in adjective-nouns. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2164–2173. Association for Computational Linguistics, 2016. doi: 10.18653/ v1/P16-1204. URL http://www.aclweb.org/ anthology/P16-1204. ↗
  37. Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella bernardi, and Roberto Zamparelli. A SICK cure for the evaluation of compositional distributional semantic models. In Nicoletta Calzolari, Khalid Choukri, Thierry Declerck, Hrafn Loftsson, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, and Stelios Piperidis (eds.), Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pp. 216–223, Reykjavik, Iceland, May 2014. European Language Resources Association (ELRA). ISBN 978-2-9517408-8-4. URL http://www.lrec-conf.org/proceedings/ lrec2014/pdf/363 Paper.pdf. ACL Anthology Identifier: L14-1314. ↗
  38. Ellie Pavlick, Travis Wolfe, Pushpendre Rastogi, Chris Callison-Burch, Mark Dredze, and Benjamin Van Durme. FrameNet+: Fast Paraphrastic Tripling of FrameNet. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pp. 408–413, Beijing, China, July 2015. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/ P15-2067. ↗
  39. Pasquale Minervini and Sebastian Riedel. Adversarially Regularising Neural NLI Models to Integrate Logical Background Knowledge. In Proceedings of the 22nd Conference on Computational Natural Language Learning (CoNLL 2018). Association for Computational Linguistics, 2018. find
  40. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. GloVe: Global Vectors for Word Representation. In Empirical Methods in Natural Language Processing (EMNLP), pp. 1532– 1543, 2014. URL http://www.aclweb.org/ anthology/D14-1162. ↗
  41. Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. A Corpus and Cloze Evaluation for Deeper Understanding of Commonsense Stories. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 839– 849, San Diego, California, June 2016. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/N16-1098. ↗
  42. Adam Poliak, Aparajita Haldar, Rachel Rudinger, J. Edward Hu, Ellie Pavlick, Aaron Steven White, and Benjamin Van Durme. Collecting Diverse Natural Language Inference Problems for Sentence Representation Evaluation. In Empirical Methods in Natural Language Processing (EMNLP), 2018a. find our card
  43. Lili Mou, Rui Men, Ge Li, Yan Xu, Lu Zhang, Rui Yan, and Zhi Jin. Natural Language Inference by Tree-Based Convolution and Heuristic Matching. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 130–136, Berlin, Germany, August 2016. Association for Computational Linguistics. URL http://anthology.aclweb.org/ P16-2022. ↗ our card
  44. Adam Poliak, Jason Naradowsky, Aparajita Haldar, Rachel Rudinger, and Benjamin Van Durme. Hypothesis Only Baselines in Natural Language Inference. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, pp. 180–191, New Orleans, Louisiana, June 2018b. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/ S18-2023. ↗
  45. Pramod Kaushik Mudrakarta, Ankur Taly, Mukund Sundararajan, and Kedar Dhamdhere. Did the Model Understand the Question? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1896–1906. Association for Computational Linguistics, 2018. URL http://aclweb.org/ anthology/P18-1176. ↗
  46. Altaf Rahman and Vincent Ng. Resolving Complex Cases of Definite Pronouns: The Winograd Schema Challenge. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pp. 777–789, Jeju Island, Korea, July 2012. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/ D12-1071. ↗ our card
  47. Ankur Parikh, Oscar Täckström, Dipanjan Das, and Jakob Uszkoreit. A Decomposable Attention Model for Natural Language Inference. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2249– 2255. Association for Computational Linguistics, Sainandan Ramakrishnan, Aishwarya Agrawal, and Stefan Lee. Overcoming language priors in visual question answering with adversarial regularization. In Advances in Neural Information Processing Systems, pp. 1548–1558, 2018. find our card
  48. Aarne Talman and Stergios Chatzikyriakidis. Neural Network Models for Natural Language Inference Fail to Capture the Semantics of Inference. CoRR, abs/1810.09774, 2018. URL http:// arxiv.org/abs/1810.09774. ↗
  49. Pushpendre Rastogi and Benjamin Van Durme. Sublinear partition estimation. arXiv preprint arXiv:1508.01596, 2015. ↗
  50. Drew Reisinger, Rachel Rudinger, Francis Ferraro, Craig Harman, Kyle Rawlins, and Benjamin Van Durme. Semantic Proto-Roles. Transactions of the Association for Computational Linguistics, 3:475– 488, 2015. ISSN 2307-387X. URL https://tacl2013.cs.columbia.edu/ojs/ index.php/tacl/article/view/674. ↗
  51. Florian Tramr, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. Ensemble Adversarial Training: Attacks and Defenses. In International Conference on Learning Representations, 2018. URL https:// openreview.net/forum?id=rkZvSe-RZ. ↗
  52. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Semantically Equivalent Adversarial Rules for Debugging NLP models. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 856–865. Association for Computational Linguistics, 2018. URL http://aclweb.org/ anthology/P18-1079. ↗
  53. Masatoshi Tsuchiya. Performance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailment. In 11th International Conference on Language Resources and Evaluation (LREC2018), 2018. find
  54. Lucy Vanderwende and William B Dolan. What syntax can contribute in the entailment task. In Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment, pp. 205–216. Springer, 2006. find
  55. Tim Rocktäschel, Edward Grefenstette, Karl Moritz Hermann, Tomas Kocisky, and Phil Blunsom. Reasoning about Entailment with Neural Attention. In International Conference on Learning Representations (ICLR), 2016. find our card
  56. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pp. 353–355, Brussels, Belgium, November 2018. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/ W18-5446. ↗ our card
  57. Roy Schwartz, Maarten Sap, Ioannis Konstas, Leila Zilles, Yejin Choi, and Noah A. Smith. The Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Task. In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pp. 15–25. Association for Computational Linguistics, 2017a. doi: 10.18653/v1/K17-1004. URL http://www.aclweb.org/anthology/K17-1004. ↗
  58. Aaron Steven White, Pushpendre Rastogi, Kevin Duh, and Benjamin Van Durme. Inference is Everything: Recasting Semantic Resources into a Unified Evaluation Framework. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 996–1005, Taipei, Taiwan, November 2017. Asian Federation of Natural Language Processing. URL http:// www.aclweb.org/anthology/I17-1100. ↗ our card
  59. Roy Schwartz, Maarten Sap, Ioannis Konstas, Leila Zilles, Yejin Choi, and Noah A. Smith. Story Cloze Task: UW NLP System. In Proceedings of LSDSem, 2017b. find
  60. Rishi Sharma, James Allen, Omid Bakhshandeh, and Nasrin Mostafazadeh. Tackling the Story Ending Biases in The Story Cloze Test. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 752–757, Melbourne, Australia, July 2018. Association for Computational Linguistics. URL http://www.aclweb.org/anthology/P18-2119. ↗
  61. Adina Williams, Nikita Nangia, and Samuel Bowman. A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 1112–1122. Association for Computational Linguistics, 2018. URL http://aclweb.org/anthology/N18-1101. ↗
  62. Rion Snow, Lucy Vanderwende, and Arul Menezes. Effectively Using Syntax for Recognizing False Entailment. In Proceedings of the Human Language Technology Conference of the NAACL, Main Conference, 2006. URL http://www.aclweb.org/ anthology/N06-1005. ↗
  63. Qile Zhu Xiaolin Li Xiaoyong Yuan, Pan He. Adversarial Examples: Attacks and Defenses for Deep Learning. arXiv preprint arXiv:1712.07107, 2017. ↗
  64. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations (ICLR), 2014. find our card
  65. Chen Zhang. Natural Language Interference from Textual Entailment to Conversation Entailment. PhD thesis, Michigan State University, East Lansing, MI, USA, 2010. AAI3435149. find
  66. Chen Zhang and Joyce Chai. Towards Conversation Entailment: An Empirical Investigation. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pp. 756– 766, Cambridge, MA, October 2010. Association for Computational Linguistics. URL https:// www.aclweb.org/anthology/D10-1074. ↗
  67. Chen Zhang and Joyce Y. Chai. What Do We Know About Conversation Participants: Experiments on Conversation Entailment. In Proceedings of the SIGDIAL 2009 Conference: The 10th Annual Meeting of the Special Interest Group on Discourse and Dialogue, SIGDIAL ’09, pp. 206–215, Stroudsburg, PA, USA, 2009. Association for Computational Linguistics. ISBN 978-1-932432-64-0. URL http://dl.acm.org/ citation.cfm?id=1708376.1708406. ↗
  68. P. Zhang, Y. Goyal, D. Summers-Stay, D. Batra, and D. Parikh. Yin and Yang: Balancing and Answering Binary Visual Questions. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5014–5022, June 2016. doi: 10.1109/ CVPR.2016.542. find
  69. Sheng Zhang, Rachel Rudinger, Kevin Duh, and Benjamin Van Durme. Ordinal Common-sense Inference. Transactions of the Association for Computational Linguistics, 5:379–395, 2017. ISSN 2307-387X. URL https://transacl.org/ojs/ index.php/tacl/article/view/1082. ↗ our card