Elamur Research
☰
home / papers / 2018 / The Natural Language Decathlon: Multitask Learning as Question Answering

The Natural Language Decathlon: Multitask Learning as Question Answering

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "arXiv — non-exclusive licence to distribute" does not allow redistributing the text itself. Original — arxiv.org/pdf/1806.08730

Summary

Most NLP progress comes from models built for one dataset, one metric and one task. The authors argue that general-purpose language systems cannot emerge that way. To test broader models, they propose decaNLP, a benchmark that combines ten public datasets. The tasks are QA, translation, summarization, NLI, sentiment, semantic role labeling, relation extraction, dialogue state tracking, semantic parsing and pronoun resolution. Every example becomes a (question, context, answer) triplet, so a natural-language question tells the model which task to do. They also present MQAN, a single network with no task-specific modules or parameters. It encodes inputs with BiLSTMs, dual coattention and self-attention. Its multi-pointer-generator decoder picks each output token from the question, from the context, or from a fixed vocabulary. Trained jointly on all ten tasks with an anti-curriculum schedule, MQAN nearly matches ten separately trained copies while using far fewer parameters in total. Pretraining on decaNLP also helps with transfer, domain adaptation and zero-shot classification. The released code and leaderboard give researchers a shared testbed for multitask, transfer and meta-learning work.

Key points

  • The ten datasets are SQuAD, IWSLT (English→German), CNN/DM, MNLI, SST, QA-SRL, QA-ZRE, WOZ, WikiSQL and MWSC. Their task metrics (nF1, BLEU, ROUGE, EM, cF1, dsEM, lfEM) are combined into one decaScore.
  • Training set sizes vary widely, from 80 examples (MWSC) to 840,000 (QA-ZRE).
  • Ablations show the multi-pointer-generator decoder is central to MQAN's multitask results, and anti-curriculum ordering adds further gains.
  • After pretraining on decaNLP, MQAN improves transfer to machine translation and NER. It also adapts to new domains for sentiment and NLI, and it can classify text zero-shot.
  • Trained on WikiSQL alone, MQAN set a new state of the art for that task. The data, training and evaluation code is released at github.com/salesforce/decaNLP.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Other

Topics: Semantic role labeling

Cited by

220 papers of our library cite it (of the 140673 whose bibliographies we read); 35 of them have a page here, 185 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1806.08730 · arXiv 1806.08730

@misc{mccann2018natural,
  title = {{The Natural Language Decathlon: Multitask Learning as Question Answering}},
  author = {McCann, Bryan and Keskar, Nitish Shirish and Xiong, Caiming and Socher, Richard},
  year = {2018},
  doi = {10.48550/arxiv.1806.08730},
  eprint = {1806.08730},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1806.08730}
}

References

127 references, 31 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. M. Andrychowicz, M. Denil, S. Gomez, M. W. Hoffman, D. Pfau, T. Schaul, and N. de Freitas. Learning to learn by gradient descent by gradient descent. In NIPS, pages 3981–3989, 2016. find
  2. J. Ba, R. Kiros, and G. E. Hinton. Layer normalization. CoRR, abs/1607.06450, 2016. find our card
  3. D. Bahdanau, K. Cho, and Y. Bengio. Neural machine translation by jointly learning to align and translate. CoRR, abs/1409.0473, 2014. find our card
  4. S. Bengio, Y. Bengio, J. Cloutier, and J. Gecsei. On the optimization of a synaptic learning rule. Optimality in Artificial and Biological Neural, pages Networks, pp. , 1992, 1992. find
  5. Y. Bengio, J. Louradour, R. Collobert, and J. Weston. Curriculum learning. In ICML, 2009. find our card
  6. J. Bos and K. Markert. Recognising textual entailment with robust logical inference. In MLCW, 2005. find
  7. S. R. Bowman, G. Angeli, C. Potts, and C. D. Manning. A large annotated corpus for learning natural language inference. In EMNLP, 2015. find
  8. R. Caruana. Multitask learning. Machine Learning, 28:41–75, 1997. find
  9. M. Cettolo, J. Niehues, S. Stüker, L. Bentivogli, R. Cattoni, and M. J. Federico. The iwslt 2016 evaluation campaign. In IWSLT, 2016. find
  10. M. X. Chen, O. Firat, A. Bapna, M. Johnson, W. Macherey, G. Foster, L. Jones, N. Parmar, M. Schuster, Z. Chen, et al. The best of both worlds: Combining recent advances in neural machine translation. arXiv preprint arXiv:1804.09849, 2018. find our card
  11. Q. Chen, X.-D. Zhu, Z.-H. Ling, D. Inkpen, and S. Wei. Natural language inference with external knowledge. CoRR, abs/1711.04289, 2017. — [arXiv:1711.04289](https://arxiv.org/abs/1711.04289) find
  12. J. Choi, K. M. Yoo, and S. goo Lee. Learning to compose task-specific tree structures. In Association for the Advancement of Artificial Intelligence, 2017. find
  13. R. Collobert and J. Weston. A unified architecture for natural language processing: deep neural networks with multitask learning. In ICML, 2008. find
  14. R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, and P. P. Kuksa. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12:2493–2537, 2011. find our card
  15. C. Condoravdi, D. Crouch, V. de Paiva, R. Stolle, and D. G. Bobrow. Entailment, intensionality and text understanding. In Proceedings of the HLT-NAACL 2003 Workshop on Text Meaning - Volume 9, HLT-NAACL-TEXTMEANING ’03, 2003. find
  16. I. Dagan, O. Glickman, and B. Magnini. The pascal recognising textual entailment challenge. In MLCW, 2005. find
  17. K. Deb. Multi-objective optimization. In Search methodologies, pages 403–449. Springer, 2014. find
  18. L. Dong and M. Lapata. Coarse-to-fine decoding for neural semantic parsing. arXiv preprint arXiv:1805.04793, 2018. find
  19. C. Fernando, D. Banarse, C. Blundell, Y. Zwols, D. Ha, A. A. Rusu, A. Pritzel, and D. Wierstra. Pathnet: Evolution channels gradient descent in super neural networks. arXiv preprint arXiv:1701.08734, 2017. find
  20. C. Finn, P. Abbeel, and S. Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In ICML, 2017. find
  21. Y. Fyodorov, Y. Winter, and N. Francez. A natural logic inference system. In In Proceedings of the 2nd Workshop on Inference in Computational Semantics (ICoS-2), 2000. find
  22. J. Gehring, M. Auli, D. Grangier, D. Yarats, and Y. Dauphin. Convolutional sequence to sequence learning. In ICML, 2017. find our card
  23. A. Gepperth and C. Karaoguz. A bio-inspired incremental learning architecture for applied perceptual problems. Cognitive Computation, 8(5):924–934, 2016. find
  24. R. Ghaeini, S. A. Hasan, V. Datla, J. Liu, K. Lee, A. Qadir, Y. Ling, A. Prakash, X. Z. Fern, and O. Farri. Dr-bilstm: Dependent reading bidirectional lstm for natural language inference. CoRR, abs/1802.05577, 2018. find
  25. A. Graves and J. Schmidhuber. Framewise phoneme classification with bidirectional lstm and other neural network architectures. Neural networks : the official journal of the International Neural Network Society, 18 5-6:602–10, 2005. find
  26. M. Grusky, M. Naaman, and Y. Artzi. Newsroom: A dataset of 1.3 million summaries with diverse extractive strategies. In NAACL-HLT, New Orleans, Louisiana, 2018. find our card
  27. J. Gu, Z. Lu, H. Li, and V. O. K. Li. Incorporating Copying Mechanism in Sequence-to-Sequence Learning. CoRR, abs/1603.06393, 2016. c. Gülçehre, S. Ahn, R. Nallapati, B. Zhou, and Y. Bengio. Pointing the Unknown Words. arXiv preprint arXiv:1603.08148, 2016. find our card
  28. K. Hashimoto, C. Xiong, Y. Tsuruoka, and R. Socher. A joint many-task model: Growing a neural network for multiple NLP tasks. In ICLR, 2016. find our card
  29. L. He, M. Lewis, and L. S. Zettlemoyer. Question-answer driven semantic role labeling: Using natural language to annotate natural language. In EMNLP, 2015. find our card
  30. L. He, K. Lee, M. Lewis, and L. S. Zettlemoyer. Deep semantic role labeling: What works and what’s next. In ACL, 2017. find our card
  31. K. M. Hermann, T. Kociský, E. Grefenstette, L. Espeholt, W. Kay, M. Suleyman, and P. Blunsom. Teaching machines to read and comprehend. In NIPS, 2015. find our card
  32. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9 8:1735–80, 1997. find
  33. S. Hochreiter, A. S. Younger, and P. R. Conwell. Learning to learn using gradient descent. In International Conference on Artificial Neural Networks, pages 87–94. Springer, 2001. find
  34. E. H. Hovy, M. P. Marcus, M. Palmer, L. A. Ramshaw, and R. M. Weischedel. Ontonotes: The 90 In HLT-NAACL, 2006. find
  35. J. Howard and S. Ruder. Fine-tuned language models for text classification. CoRR, abs/1801.06146, 2018. find
  36. M. Hu, Y. Peng, Z. Huang, X. Qiu, F. Wei, and M. Zhou. Reinforced mnemonic reader for machine reading comprehension. CoRR, abs/1705.02798, 2018. find
  37. H.-Y. Huang, C. Zhu, Y. Shen, and W. Chen. Fusionnet: Fusing via fully-aware attention with application to machine comprehension. CoRR, abs/1711.07341, 2017. — [arXiv:1711.07341](https://arxiv.org/abs/1711.07341) find
  38. P.-S. Huang, C. Wang, R. Singh, W.-t. Yih, and X. He. Natural language to structured query generation via meta-learning. arXiv preprint arXiv:1803.02400, 2018. find
  39. J. Im and S. Cho. Distance-based self-attention network for natural language inference. CoRR, abs/1712.02047, 2017. — [arXiv:1712.02047](https://arxiv.org/abs/1712.02047) find
  40. R. Johansson and P. Nugues. Dependency-based semantic role labeling of propbank. In EMNLP, 2008. find our card
  41. M. Johnson, M. Schuster, Q. V. Le, M. Krikun, Y. Wu, Z. Chen, N. Thorat, F. B. Viégas, M. Wattenberg, G. S. Corrado, M. Hughes, and J. Dean. Google’s multilingual neural machine translation system: Enabling zero-shot translation. TACL, 5:339–351, 2017. find
  42. M. Joshi, E. Choi, D. S. Weld, and L. S. Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In ACL, 2017. find our card
  43. L. Kaiser, A. N. Gomez, N. Shazeer, A. Vaswani, N. Parmar, L. Jones, and J. Uszkoreit. One model to learn them all. CoRR, abs/1706.05137, 2017. — [arXiv:1706.05137](https://arxiv.org/abs/1706.05137) find
  44. R. Kemker, A. Abitino, M. McClure, and C. Kanan. Measuring catastrophic forgetting in neural networks. CoRR, abs/1708.02072, 2017. find
  45. S. Kim, J.-H. Hong, I. Kang, and N. Kwak. Semantic sentence matching with densely-connected recurrent and co-attentive information. arXiv preprint arXiv:1805.11360, 2018. — [arXiv:1805.11360](https://arxiv.org/abs/1805.11360) find
  46. J. Kirkpatrick, R. Pascanu, N. C. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, D. Hassabis, C. Clopath, D. Kumaran, and R. Hadsell. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences of the… find
  47. R. Kiros, Y. Zhu, R. Salakhutdinov, R. S. Zemel, R. Urtasun, A. Torralba, and S. Fidler. Skip-thought vectors. In NIPS, 2015. find
  48. D. Kotzias, M. Denil, N. De Freitas, and P. Smyth. From group to individual labels using deep features. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 597–606. ACM, 2015. find
  49. A. Kumar, O. Irsoy, P. Ondruska, M. Iyyer, J. Bradbury, I. Gulrajani, V. Zhong, R. Paulus, and R. Socher. Ask me anything: Dynamic memory networks for natural language processing. In ICML, 2016. find our card
  50. S. Lee, J. Kim, J. Jun, J. Ha, and B. Zhang. Overcoming catastrophic forgetting by incremental moment matching. In Advances in Neural Information Processing Systems, pages 4655–4665, 2017. find
  51. H. J. Levesque, E. Davis, and L. Morgenstern. The winograd schema challenge. In Aaai spring symposium: Logical formalizations of commonsense reasoning, volume 46, page 47, 2011. find
  52. O. Levy, M. Seo, E. Choi, and L. S. Zettlemoyer. Zero-shot relation extraction via reading comprehension. In CoNLL, 2017. find
  53. C.-Y. Lin. Rouge: A package for automatic evaluation of summaries. In Proceedings of ACL workshop on Text Summarization, 2004. find
  54. R. Liu, W. Wei, W. Mao, and M. Chikina. Phase conductor on multi-layered attentions for machine comprehension. CoRR, abs/1710.10504, 2017a. — [arXiv:1710.10504](https://arxiv.org/abs/1710.10504) find
  55. X. Liu, Y. Shen, K. Duh, and J. Gao. Stochastic answer networks for machine reading comprehension. CoRR, abs/1712.03556, 2017b. find
  56. M.-T. Luong, Q. V. Le, I. Sutskever, O. Vinyals, and L. Kaiser. Multi-task sequence to sequence learning. CoRR, abs/1511.06114, 2015a. find our card
  57. T. Luong, H. Pham, and C. D. Manning. Effective approaches to attention-based neural machine translation. In EMNLP, 2015b. find
  58. B. MacCartney and C. D. Manning. An extended model of natural logic. In IWCS, 2009. find
  59. A. Mallya and S. Lazebnik. Packnet: Adding multiple tasks to a single network by iterative pruning. arXiv preprint arXiv:1711.05769, 2017. find
  60. D. Marcheggiani, A. Frolov, and I. Titov. A simple and accurate syntax-agnostic neural model for dependency-based semantic role labeling. In CoNLL, 2017. find our card
  61. B. McCann, J. Bradbury, C. Xiong, and R. Socher. Learned in translation: Contextualized word vectors. In NIPS, 2017. find our card
  62. M. McCloskey and N. J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pages 109–165. Elsevier, 1989. find
  63. S. Merity, C. Xiong, J. Bradbury, and R. Socher. Pointer sentinel mixture models. ICLR, 2017. find our card
  64. T. Mikolov, K. Chen, G. S. Corrado, and J. Dean. Efficient estimation of word representations in vector space. CoRR, abs/1301.3781, 2013a. find
  65. T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In NIPS, 2013b. find
  66. S. Min, M. J. Seo, and H. Hajishirzi. Question answering through transfer learning from large fine-grained supervision data. In ACL, 2017. find
  67. T. M. Mitchell. The need for biases in learning generalizations. 1980. find
  68. N. Mrkšić, D. O. Séaghdha, T.-H. Wen, B. Thomson, and S. Young. Neural belief tracker: Data-driven dialogue state tracking. arXiv preprint arXiv:1606.03777, 2016. find
  69. V. Nair and G. E. Hinton. Rectified linear units improve restricted boltzmann machines. In ICML, 2010. find
  70. R. Nallapati, B. Zhou, C. N. dos Santos, Çaglar Gülçehre, and B. Xiang. Abstractive text summarization using sequence-to-sequence RNNs and beyond. In CoNLL, 2016. find
  71. B. Pan, H. Li, Z. Zhao, B. Cao, D. Cai, and X. He. Memen: Multi-layer embedding with memory networks for machine comprehension. CoRR, abs/1707.09098, 2017. — [arXiv:1707.09098](https://arxiv.org/abs/1707.09098) find
  72. K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu. Bleu: a method for automatic evaluation of machine translation. In ACL, 2002. find our card
  73. R. Pasunuru and M. Bansal. Multi-reward reinforced summarization with saliency and entailment. In NAACL, 2018. find
  74. R. Pasunuru, H. Guo, and M. Bansal. Towards improving abstractive summarization via entailment generation. In NFiS@EMNLP, 2017. find
  75. R. Paulus, C. Xiong, and R. Socher. A deep reinforced model for abstractive summarization. arXiv preprint arXiv:1705.04304, 2017. find our card
  76. J. Pennington, R. Socher, and C. D. Manning. Glove: Global vectors for word representation. In EMNLP, 2014. find
  77. M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. S. Zettlemoyer. Deep contextualized word representations. CoRR, abs/1802.05365, 2018. find
  78. A. Poliak, Y. Belinkov, J. Glass, and B. Van Durme. On the evaluation of semantic phenomena in neural machine translation using natural language inference. arXiv preprint arXiv:1804.09779, 2018a. find
  79. A. Poliak, A. Haldar, R. Rudinger, J. E. Hu, E. Pavlick, A. S. White, and B. Van Durme. Towards a unified natural language inference framework to evaluate sentence representations. arXiv preprint arXiv:1804.08207, 2018b. find our card
  80. V. Punyakanok, D. Roth, and W. tau Yih. The importance of syntactic parsing and inference in semantic role labeling. Computational Linguistics, 34:257–287, 2008. find our card
  81. A. Radford, R. Józefowicz, and I. Sutskever. Learning to generate reviews and discovering sentiment. CoRR, abs/1704.01444, 2017. find our card
  82. P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang. Squad: 100, 000+ questions for machine comprehension of text. In EMNLP, 2016. find our card
  83. P. Ramachandran, P. J. Liu, and Q. V. Le. Unsupervised pretraining for sequence to sequence learning. In EMNLP, 2017. find
  84. R. M. Ratcliff. Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychological review, 97 2:285–308, 1990. find
  85. A. Robins. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Science, 7(2):123–146, 1995. find
  86. S. Ruder, J. Bingel, I. Augenstein, and A. Sogaard. Sluice networks: Learning what to share between loosely related tasks. CoRR, abs/1705.08142, 2017. — [arXiv:1705.08142](https://arxiv.org/abs/1705.08142) find
  87. S. Salant and J. Berant. Contextualized word representations for reading comprehension. CoRR, abs/1712.03609, 2017. find
  88. A. Santoro, S. Bartunov, M. Botvinick, D. Wierstra, and T. Lillicrap. Meta-learning with memory-augmented neural networks. In International conference on machine learning, pages 1842–1850, 2016. find
  89. J. Schmidhuber. Evolutionary principles in self-referential learning, or on learning how to learn: the meta-meta-... hook. PhD thesis, Technische Universität München, 1987. find
  90. J. Schmidhuber. Learning to control fast-weight memories: An alternative to dynamic recurrent networks. Neural Computation, 4(1):131–139, 1992. find
  91. A. See, P. J. Liu, and C. D. Manning. Get to the point: Summarization with pointer-generator networks. In ACL, 2017. find
  92. R. Sennrich, A. Birch, A. Currey, U. Germann, B. Haddow, K. Heafield, A. V. M. Barone, and P. Williams. The university of edinburgh’s neural mt systems for wmt17. In WMT, 2017. find
  93. M. Seo, A. Kembhavi, A. Farhadi, and H. Hajishirzi. Bidirectional attention flow for machine comprehension. ICLR, 2017. find our card
  94. T. Shen, T. Zhou, G. Long, J. Jiang, S. Wang, and C. Zhang. Reinforced self-attention network: a hybrid of hard and soft attention for sequence modeling. CoRR, abs/1801.10296, 2018. — [arXiv:1801.10296](https://arxiv.org/abs/1801.10296) find
  95. R. Socher, A. Perelygin, J. Wu, J. Chuang, C. Manning, A. Ng, and C. Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In EMNLP, 2013. find
  96. I. Sutskever, O. Vinyals, and Q. V. Le. Sequence to sequence learning with neural networks. In NIPS, 2014. find
  97. J. Suzuki and M. Nagata. Cutting-off redundant repeating generations for neural abstractive summarization. In EACL, 2017. find
  98. K. S. Tai, R. Socher, and C. D. Manning. Improved semantic representations from tree-structured long short-term memory networks. In ACL, 2015. find
  99. Z. Tan, M. Wang, J. Xie, Y. Chen, and X. Shi. Deep semantic role labeling with self-attention. CoRR, abs/1712.01586, 2017. find
  100. Y. Tay, L. A. Tuan, and S. C. Hui. A compare-propagate architecture with alignment factorization for natural language inference. CoRR, abs/1801.00102, 2017. — [arXiv:1801.00102](https://arxiv.org/abs/1801.00102) find
  101. S. Thrun. Lifelong learning algorithms. In Learning to learn, pages 181–209. Springer, 1998. find
  102. S. Thrun and L. Pratt. Learning to learn: Introduction and overview. In Learning to learn, pages 3–17. Springer, 1998. find
  103. A. Trischler, T. Wang, X. Yuan, J. Harris, A. Sordoni, P. Bachman, and K. Suleman. Newsqa: A machine comprehension dataset. In Rep4NLP@ACL, 2017. find our card
  104. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. Attention is all you need. In NIPS, 2017. find
  105. R. Vilalta and Y. Drissi. A perspective view and survey of meta-learning. Artificial Intelligence Review, 18(2):77–95, 2002. find
  106. O. Vinyals, M. Fortunato, and N. Jaitly. Pointer networks. In NIPS, 2015. find
  107. A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461, 2018. find our card
  108. S. Wang and J. Jiang. Machine comprehension using Match-LSTM and answer pointer. In ICLR, 2017. find our card
  109. W. Wang, N. Yang, F. Wei, B. Chang, and M. Zhou. Gated self-matching networks for reading comprehension and question answering. In ACL, 2017a. find
  110. Z. Wang, W. Hamza, and R. Florian. Bilateral multi-perspective matching for natural language sentences. In IJCAI, 2017b. find our card
  111. D. Weissenborn, G. Wiese, and L. Seiffe. Making neural qa as simple as possible but not simpler. In CoNLL, 2017. find
  112. T.-H. Wen, D. Vandyke, N. Mrksic, M. Gasic, L. M. Rojas-Barahona, P.-H. Su, S. Ultes, and S. Young. A network-based end-to-end trainable task-oriented dialogue system. arXiv preprint arXiv:1604.04562, 2016. find
  113. A. Williams, N. Nangia, and S. R. Bowman. A broad-coverage challenge corpus for sentence understanding through inference. CoRR, abs/1704.05426, 2017. find
  114. T. Winograd. Understanding natural language. Cognitive psychology, 3(1):1–191, 1972. find
  115. Y. Wu, M. Schuster, Z. Chen, Q. V. Le, M. Norouzi, W. Macherey, M. Krikun, Y. Cao, Q. Gao, K. Macherey, J. Klingner, A. Shah, M. Johnson, X. Liu, L. Kaiser, S. Gouws, Y. Kato, T. Kudo, H. Kazawa, K. Stevens, G. Kurian, N. Patil, W. Wang, C. Young, J. Smith, J. Riesa, A. Rudnick, O. Vinyals, G. S. Co… find
  116. C. Xiong, S. Merity, and R. Socher. Dynamic Memory Networks for Visual and Textual Question Answering. In ICML, 2016. find
  117. C. Xiong, V. Zhong, and R. Socher. Dynamic coattention networks for question answering. ICLR, 2017. find our card
  118. C. Xiong, V. Zhong, and R. Socher. Dcn+: Mixed objective and deep residual coattention for question answering. ICLR, 2018. find
  119. A. W. Yu, D. Dohan, M.-T. Luong, R. Zhao, K. Chen, M. Norouzi, and Q. V. Le. Qanet: Combining local convolution with global self-attention for reading compre- hension. In ICLR, 2018a. find
  120. H. Yu and T. Munkhdalai. Neural semantic encoders. Proceedings of the conference. Association for Computational Linguistics. Meeting, 1:397–407, 2017a. find
  121. H. Yu and T. Munkhdalai. Neural tree indexers for text understanding. ACL, 2017b. find
  122. T. Yu, Z. Li, Z. Zhang, R. Zhang, and D. Radev. Typesql: Knowledge-based type-aware neural text-to-sql generation. arXiv preprint arXiv:1804.09769, 2018b. find
  123. Y. Yu, W. Zhang, K. S. Hasan, M. Yu, B. Xiang, and B. Zhou. End-to-end reading comprehension with dynamic answer chunk ranking. CoRR, abs/1610.09996, 2016. — [arXiv:1610.09996](https://arxiv.org/abs/1610.09996) find
  124. V. Zhong, C. Xiong, and R. Socher. Seq2sql: Generating structured queries from natural language using reinforcement learning. CoRR, abs/1709.00103, 2017. find
  125. V. Zhong, C. Xiong, and R. Socher. Global-locally self-attentive dialogue state tracker. arXiv preprint arXiv:1805.09655, 2018. — [arXiv:1805.09655](https://arxiv.org/abs/1805.09655) find
  126. J. Zhou and W. Xu. End-to-end learning of semantic role labeling using recurrent neural networks. In ACL, 2015. find
  127. B. Zoph, D. Yuret, J. May, and K. Knight. Transfer learning for low-resource neural machine translation. In EMNLP, 2016. find