Elamur Research⌕
☰
home / papers / 2017 / Reinforced Mnemonic Reader for Machine Reading Comprehension

Reinforced Mnemonic Reader for Machine Reading Comprehension

Paper · 2017 · Minghao Hu, Yuxing Peng, Zhen Huang et al.

Summary

The paper addresses extractive machine reading comprehension (MRC), where a model must locate the answer span to a question inside a context paragraph. The authors identify two weaknesses in earlier attentive readers. First, multi-round alignment models compute each attention layer without knowing what previous layers attended to. As a result, layers either repeat each other (attention redundancy) or miss important tokens (attention deficiency). Second, self-critical RL training (SCST) can push the model away from partially correct answers whenever the greedy baseline scores higher than the sample, which they call the convergence suppression problem.

The proposed Reinforced Mnemonic Reader addresses both issues. Its reattention mechanism stores earlier attention distributions and uses their overlap to adjust the current similarity scores. Its dynamic-critical RL (DCRL) samples answers both randomly and greedily, then treats the higher-scoring answer as the reward and the other as the baseline, so the normalized reward is never negative.

On SQuAD, the model achieved state-of-the-art results at the time. On adversarial SQuAD, it beat previous systems by more than 6 points. This shows that refining attention and stabilizing RL training improve both accuracy and robustness.

Key points

  • On the SQuAD leaderboard, the single model scored 79.5% EM and 86.6% F1, and the ensemble reached 82.3% EM and 88.5% F1.
  • On the adversarial AddSent and AddOneSent sets (Jia and Liang, 2017), the model outperformed earlier approaches by over 6% in both EM and F1.
  • The architecture stacks 3 aligning blocks that combine interactive and self-alignment. In ablations, 2 blocks hurt results slightly, 4 blocks made almost no difference, and 5 blocks led to a clear drop, which the authors attribute to network degradation.
  • KL-divergence analysis on the SQuAD dev set supports the reattention claims:
    • Redundancy decreased: the divergence between E¹ and E² attention grew from 0.695 to 0.866.
    • Deficiency decreased: the distance to the ensemble-averaged attention fell from 0.650 to 0.568 for E², and from 0.536 to 0.482 for E³.
  • In qualitative comparisons, DCRL finds tighter answer boundaries than SCST, which often selects overly long phrases.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Fudan UniversityShanghai · China202 works · 22,509 citations🏢Microsoft Research AsiaBeijing · China82 works · 61,484 citations🎓National University of Defense TechnologyChangsha · China20 works · 1,339 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 1Research subjects and languages the work is about.Question answering · 487 works

Cite

Cite as Minghao Hu, Yuxing Peng, Zhen Huang, Xipeng Qiu, Furu Wei and Ming Zhou. 2017. Reinforced Mnemonic Reader for Machine Reading Comprehension. arXiv:1705.02798.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.1705.02798 · arXiv 1705.02798

@misc{hu2017reinforced,
  title = {{Reinforced Mnemonic Reader for Machine Reading Comprehension}},
  author = {Minghao Hu, Yuxing Peng, Zhen Huang, Xipeng Qiu, Furu Wei and Ming Zhou},
  year = {2017},
  doi = {10.48550/arxiv.1705.02798},
  eprint = {1705.02798},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1705.02798}
}

References

28 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. Reading wikipedia to answer open-domain questions. arXiv preprint arXiv:1704.00051, 2017.
  2. Yoshua Bengio Dzmitry Bahdanau, Kyunghyun Cho. Neural machine translation by jointly learning to align and translate. In Proceedings of ICLR, 2015.
  3. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT Press, 2016. find
  4. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of CVPR, pages 770–778, 2016.
  5. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, , and Phil Blunsom. Teaching machines to read and comprehend. In Proceedings of NIPS, 2015.
  6. Felix Hill, Antoine Bordes, Sumit Chopra, and Jason Weston. The goldilocks principle: Reading children’s books with explicit memory representations. In Proceedings of ICLR, 2016. — The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations
  7. Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735—-1780, 1997. find
  8. Hsin-Yuan Huang, Chenguang Zhu, Yelong Shen, and Weizhu Chen. Fusionnet: Fusing via fully-aware attention with application to machine comprehension. arXiv preprint arXiv:1711.07341, 2017. find
  9. Robin Jia and Percy Liang. Adversarial examples for evaluating reading comprehension systems. In Proceedings of EMNLP, 2017.
  10. Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. arXiv preprint arXiv:1705.07115, 2017. — [arXiv:1705.07115](https://arxiv.org/abs/1705.07115) find
  11. Diederik P. Kingma and Lei Jimmy Ba. Adam: A method for stochastic optimization. In CoRR, abs/1412.6980, 2014.
  12. Rui Liu, Junjie Hu, Wei Wei, Zi Yang, and Eric Nyberg. Structural embedding of syntactic trees for machine comprehension. arXiv preprint arXiv:1703.00572, 2017. find
  13. Xiaodong Liu, Yelong Shen, Kevin Duh, and Jianfeng Gao. Stochastic answer networks for machine reading comprehension. arXiv preprint arXiv:1712.03556, 2017. find
  14. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. Glove: Global vectors for word representation. In Proceedings of EMNLP, 2014.
  15. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word prepresentations. In Proceedings of NAACL, 2018. find
  16. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of EMNLP, 2016.
  17. Steven J Rennie, Etienne Marcheret, Youssef Mroueh, Jarret Ross, and Vaibhava Goel. Self-critical sequence training for image captioning. arXiv preprint arXiv:1612.00563, 2016.
  18. Minjoon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hananneh Hajishirzi. Bidirectional attention flow for machine comprehension. In Proceedings of ICLR, 2017.
  19. Yelong Shen, Po-Sen Huang, Jianfeng Gao, and Weizhu Chen. Reasonet: Learning to stop reading in machine comprehension. arXiv preprint arXiv:1609.05284, 2016. — [arXiv:1609.05284](https://arxiv.org/abs/1609.05284) find
  20. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, pages 1929–1958, 2014.
  21. RupeshKumar Srivastava, Klaus Greff, and Jurgen Schmidhuber. Highway networks. arXiv preprint arXiv:1505.00387, 2015.
  22. Richard S. Sutton and Andrew G. Barto. Reinforcement learning: An introduction. MIT Press, 1998. find
  23. Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. Pointer networks. In Proceedings of NIPS, 2015. find
  24. Shuohang Wang and Jing Jiang. Machine comprehension using match-lstm and answer pointer. In Proceedings of ICLR, 2017.
  25. Wenhui Wang, Nan Yang, Furu Wei, Baobao Chang, and Ming Zhou. Gated self-matching networks for reading comprehension and question answering. In Proceedings of ACL, 2017.
  26. Dirk Weissenborn, Georg Wiese, and Laura Seiffe. Making neural qa as simple as possible but not simpler. In Proceedings of CoNLL, pages 271–280, 2017.
  27. Caiming Xiong, Victor Zhong, and Richard Socher. Dcn+: Mixed objective and deep residual coattention for question answering. arXiv preprint arXiv:1711.00106, 2017. find
  28. Caiming Xiong, Victor Zhong, and Richard Socher. Dynamic coattention networks for question answering. In Proceedings of ICLR, 2017.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hu-2017-reinforced-mnemonic-reader-machine-reading/