Elamur Research⌕
☰
home / papers / 2018 / Answer-focused and Position-aware Neural Question Generation

Answer-focused and Position-aware Neural Question Generation

Paper · 2018 · Xingwu Sun, Jing Liu, Yajuan Lyu et al.

Summary

The paper tackles neural question generation: given a passage and a target answer span, the system must produce a fitting question. The authors reimplemented the strongest existing approach, a sequence-to-sequence model with copying and rich input features. Manual inspection of its output revealed two recurring errors. The question word often clashed with the answer type, for example "why" where "when" was needed. The model also tended to copy passage words located far from the answer instead of nearby relevant ones.

To fix these errors, they extend a feature-enriched pointer-generator in two ways:

  • Answer-focused component: feeds an answer embedding directly into the choice of the interrogative word.
  • Position-aware component: represents each context word's relative distance to the answer as a position embedding, which biases the copy attention toward nearby tokens.

A hybrid model combines both components. Experiments on SQuAD and MARCO show that the combination clearly beats the baseline and the previous state of the art. This matters for applications such as building reading-comprehension exercises, dialogue agents that ask questions, and creating QA datasets.

Key points

  • Error analysis: in 130 sampled baseline questions, 54 were judged poor. Of these, 20 (37.04%) had a mismatched question word and 11 (20.37%) copied distant, irrelevant context.
  • Baseline: a pointer-generator (See et al., 2017) with a bidirectional LSTM encoder and the features from Zhou et al. (2017): NE, POS, and BIO answer-position tags.
  • Answer-focused fixes: the answer-focused model repaired 7 of the 20 question-word errors, mostly cases where the answer itself reveals its type.
  • OOV copying: the position-aware model raised the average precision of copied OOV words from 21.22% to 22.79%. It raised average recall from 18.87% to 20.62%.
  • Evaluation data: SQuAD and MARCO.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢BaiduBeijing · China40 works · 14,627 citations🏛️Chinese Academy of SciencesBeijing · China68 works · 22,753 citations🔬Institute of Computing TechnologyBeijing · China18 works · 3,078 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Question answering · 242 works

Cite

Cite as Xingwu Sun, Jing Liu, Yajuan Lyu, Wei He, Yanjun Ma and Shi Wang. 2018. Answer-focused and Position-aware Neural Question Generation. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d18-1427

@inproceedings{sun2018answer,
  title = {{Answer-focused and Position-aware Neural Question Generation}},
  author = {Xingwu Sun, Jing Liu, Yajuan Lyu, Wei He, Yanjun Ma and Shi Wang},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1427},
  url = {https://doi.org/10.18653/v1/d18-1427}
}

References

15 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
  2. Michael Heilman and Noah A Smith. 2009. Question generation via overgenerating transformations and ranking. Technical report, CARNEGIE-MELLON UNIV PITTSBURGH PA LANGUAGE TECH-NOLOGIES INST. find
  3. Michael Heilman and Noah A Smith. 2010. Good question! statistical ranking for question generation. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 609–617. Association for Computational Linguistics. find
  4. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268.
  5. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics.
  6. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1532– 1543.
  7. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250.
  8. Abigail See, Peter J Liu, and Christopher D Manning. 2017. Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368. find
  9. Iulian Vlad Serban, Alberto Garcı́a-Durán, Caglar Gulcehre, Sungjin Ahn, Sarath Chandar, Aaron Courville, and Yoshua Bengio. 2016. Generating factoid questions with recurrent neural networks: The 30m factoid question-answer corpus. arXiv preprint arXiv:1603.06807. find
  10. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1):1929–1958. find
  11. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104–3112. find
  12. Duyu Tang, Nan Duan, Tao Qin, and Ming Zhou. 2017. Question answering and question generation as dual tasks. arXiv preprint arXiv:1706.02027. find
  13. Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, and Jun Zhao. 2014. Relation classification via convolutional deep neural network. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 2335–2344.
  14. Yuhao Zhang, Victor Zhong, Danqi Chen, Gabor Angeli, and Christopher D Manning. 2017. Position-aware attention and supervised data improve slot filling. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 35–45. find
  15. Qingyu Zhou, Nan Yang, Furu Wei, Chuanqi Tan, Hangbo Bao, and Ming Zhou. 2017. Neural question generation from text: A preliminary study. In National CCF Conference on Natural Language Processing and Chinese Computing, pages 662–671. Springer. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sun-2018-answer-focused-position-aware-neural/