Elamur Research⌕
☰
home / papers / 2018 / Paragraph-level Neural Question Generation with Maxout Pointer and Gated Self-attention Networks

Paragraph-level Neural Question Generation with Maxout Pointer and Gated Self-attention Networks

Summary

The paper addresses answer-aware question generation (QG). Given a passage and an answer span inside it, the system must produce a question that the span answers. Earlier neural sequence-to-sequence QG systems mostly worked with one or two sentences. Feeding them a whole paragraph tended to lower quality, even though some questions depend on context outside the answer sentence.

The authors extend an attention-based encoder-decoder (LSTM, Luong attention) with three components:

  • Answer tagging: input embeddings are marked to show whether each word belongs to the answer.
  • Gated self-attention encoder: the passage is matched against itself, and a learned gate blends this self-matching information with the original encoding.
  • Maxout pointer: a variant of the copy mechanism used in the decoder.

The model is evaluated on SQuAD and MS MARCO. With sentence inputs it already beats prior systems. It improves further when given full paragraphs, raising the state-of-the-art BLEU 4 on SQuAD from about 13.9 to 16.3. The authors present it as the first QG model to gain substantially from paragraph-level input over sentence-level input.

Key points

  • About 20% of SQuAD questions need paragraph-level information, according to Du et al., 2017, which motivates long-context QG.
  • SQuAD split 1, BLEU 4: Du et al. scored 12.28 and Song et al. 13.98. The proposed model scored 15.32 with sentence input and 16.38 with paragraph input. METEOR reached 20.25 and ROUGE-L 44.48.
  • SQuAD split 2, BLEU 4: Zhou et al. scored 13.27 and Song et al. 13.91. The proposed model scored 15.82 with sentence input and 16.85 with paragraph input.
  • MS MARCO, BLEU 4: the model scored 16.02 with sentence input and 17.24 with paragraph input. The baselines were Duan et al. at 11.46 and Du et al. at 10.46.
  • The best configuration is named s2s-a-ct-mp-gsa. In the case studies, paragraph context lets the model recover names and resolve coreferences that are missing from the answer sentence. Rich features and policy-gradient training are left as complementary future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Topics: Machine translation

Cited by

95 papers of our library cite it (of the 140884 whose bibliographies we read); 5 of them have a page here, 90 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d18-1424

@inproceedings{zhao2018paragraph,
  title = {{Paragraph-level Neural Question Generation with Maxout Pointer and Gated Self-attention Networks}},
  author = {Zhao, Yao and Ni, Xiaochuan and Ding, Yuanyuan and Ke, Qifa},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1424},
  url = {https://doi.org/10.18653/v1/d18-1424}
}

References

34 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Stan Szpakowicz Marie-Francine Moens, editor, Text Summarization Branches Out: Proceedings of the ACL-04 Workshop. Association for Computational Linguistics. find
  2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. find our card
  3. Nan Duan, Duyu Tang, Peng Chen, and Ming Zhou. 2017. Question generation for question answering. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. find our card
  4. Yichen Gong and Samuel R Bowman. 2017. Ruminating reader: Reasoning with gated multi-hop attention. arXiv preprint arXiv:1704.07415. find
  5. Ian J Goodfellow, David Warde-Farley, Mehdi Mirza, Aaron Courville, and Yoshua Bengio. 2013. Maxout networks. arXiv preprint arXiv:1302.4389. — [arXiv:1302.4389](https://arxiv.org/abs/1302.4389) find
  6. Jiatao Gu, Zhengdong Lu, Hang Li, and Victor OK Li. 2016. Incorporating copying mechanism in sequence-to-sequence learning. arXiv preprint arXiv:1603.06393. find our card
  7. Michael Heilman and Noah A Smith. 2010. Good question! statistical ranking for question generation. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 609–617. Association for Computational Linguistics. find
  8. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. find
  9. Igor Labutov, Sumit Basu, and Lucy Vanderwende. 2015. Deep questions without deep understanding. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, page 889898. Association for Computat… find
  10. Jiwei Li and Dan Jurafsky. 2016. Mutual information and diverse decoding improve neural machine translation. arXiv preprint arXiv:1601.00372. — [arXiv:1601.00372](https://arxiv.org/abs/1601.00372) find
  11. Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555. find
  12. Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015. Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025. find
  13. Michael Denkowski and Alon Lavie. 2014. Meteor universal: language specific translation evaluation for any target language. In Proceedings of the Ninth Workshop on Statistical Machine Translation. Association for Computational Linguistics. find our card
  14. Karen Mazidi and Rodney D. Nielsen. 2014. Linguistic considerations in automatic question generation. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, page 321326. Association for Computational Linguistics. find
  15. Yurii Nesterov. 1983. A method for unconstrained convex minimization problem with the rate of convergence o (1/kˆ 2). In Doklady AN USSR, volume 269, pages 543–547. find
  16. Sandeep Subramanian, Tong Wang, Xingdi Yuan, and Adam Trischler. 2017. Neural models for key phrase detection and question generation. arXiv preprint arXiv:1706.04560. — [arXiv:1706.04560](https://arxiv.org/abs/1706.04560) find
  17. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. Ms marco: A human generated machine reading comprehension dataset. In Proceedings of 30th Conference on Neural Information Processing Systems (NIPS). find our card
  18. Duyu Tang, Nan Duan, Tao Qin, Zhao Yan, and Ming Zhou. 2017. Question answering and question generation as dual tasks. arXiv preprint arXiv:1706.02027. — [arXiv:1706.02027](https://arxiv.org/abs/1706.02027) find
  19. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of 40th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics. find our card
  20. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543. Association for Computational Linguistics. find
  21. Ning Qian. 1999. On the momentum term in gradient descent learning algorithms. Neural networks, 12(1):145–151. find
  22. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP). find our card
  23. Vasile Rus, Brendan Wyse, Paul Piwek, Mihai Lintean, Svetlana Stoyanchev, and Cristian Moldovan. 2010. The first question generation shared task evaluation challenge. In Proceedings of the 6th International Natural Language Generation Conference, pages 251–257. Association for Computational Linguist… find
  24. Abigail See, Peter J Liu, and Christopher D Manning. 2017. Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368. find
  25. Shikhar Sharma, Layla El Asri, Hannes Schulz, and Jeremie Zumer. 2017. Relevance of unsupervised metrics in task-oriented dialogue for evaluating natural language generation. CoRR, abs/1706.09799. — [arXiv:1706.09799](https://arxiv.org/abs/1706.09799) find
  26. Heung-Yeung Shum, Xiaodong He, and Di Li. 2018. From eliza to xiaoice: challenges and opportunities with social chatbots. arXiv preprint arXiv:1801.01957. — [arXiv:1801.01957](https://arxiv.org/abs/1801.01957) find
  27. Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. 2016. Modeling coverage for neural machine translation. In Association for computation linguistics. find our card
  28. Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015. Pointer networks. In Neural Information Processing Systems. find
  29. Tong Wang, Xingdi (Eric) Yuan, and Adam Trischler. 2017a. A joint model for question answering and question generation. In Learning to generate natural language workshop, ICML 2017. find
  30. Wenhui Wang, Nan Yang, Furu Wei, Baobao Chang, and Ming Zhou. 2017b. Gated self-matching networks for reading comprehension and question answering. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 189–198. find our card
  31. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, ukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Ni… find
  32. Xingdi Yuan, Tong Wang, Caglar Gulcehre, Alessandro Sordoni, Philip Bachman, Sandeep Subramanian, Saizheng Zhang, and Adam Trischler. 2017. Machine comprehension by text-to-text neural question generation. arXiv preprint arXiv:1705.02012. find
  33. Qingyu Zhou, Nan Yang, Furu Wei, Chuanqi Tan, Hangbo Bao, and Ming Zhou. 2017. Neural question generation from text: A preliminary study. In National CCF Conference on Natural Language Processing and Chinese Computing, pages 662–671. Springer. find
  34. Linfeng Song, Zhiguo Wang, and Wael Hamza. 2017. A unified auery-based generative model for question generation and question answering. arXiv preprint arXiv:1709.01058. — [arXiv:1709.01058](https://arxiv.org/abs/1709.01058) find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhao-2018-paragraph-level-neural-question-generation/