Elamur Research⌕
☰
home / papers / 2020 / Incorporating Syntax and Frame Semantics in Neural Network for Machine Reading Comprehension

Incorporating Syntax and Frame Semantics in Neural Network for Machine Reading Comprehension

Summary

The paper targets multiple-choice machine reading comprehension (MRC). The authors argue that neural readers, including fine-tuned BERT, often fail to capture deeper meaning, and that no prior neural MRC work had combined syntactic and semantic cues. Their model, SS-MRC, adds two derived views of the input text. In the first, each word is replaced by its dependency label from Stanford CoreNLP. In the second, each word is replaced by FrameNet frames and frame elements produced by the SEMAFOR parser. BERT encodes the original passage, question and answer candidate, while a Bi-LSTM-based fusion module encodes the two auxiliary views. The answer predictor then combines both representations. Three fusion strategies are compared, and the best one merges syntactic and frame information token by token. On MCTest, SS-MRC slightly outperforms ten earlier systems, both feature-based and neural. Ablations show that each knowledge source helps, with frame semantics contributing more. The work suggests that explicit linguistic structure can still add value on top of pretrained encoders, at least on small datasets.

Key points

  • Evaluation uses MCTest-160 and MCTest-500. SS-MRC reaches 87.2% and 86.7% accuracy, compared with 82.5% and 80.9% for plain BERT. The margin over the strongest baselines is small, for example 86.6% for XLNet+DCMN+ on MCTest-500.
  • The three fusion variants are:
    • Siamese-based (SFM): separate Bi-LSTMs for each view, scoring 86.2/84.5.
    • Mixed-based (MFM): the two views concatenated into one sequence, scoring 86.6/85.0.
    • Location-wise (LFM): per-token concatenation of both views, scoring best at 87.2/86.7.
  • In the ablation, removing syntax gives 86.1/84.2, which matches the earlier frame-semantics model FSR. Removing frame semantics gives 85.4/83.0.
  • The tools and resources used are Stanford CoreNLP (dependency parsing), SEMAFOR and FrameNet (frames and frame elements), BERT, and Bi-LSTM. The fusion module is described as backbone-agnostic.
  • Training ran on a single Nvidia P100 (16 GB) with Adam, a batch size of 8, and a learning rate of 5e-5.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cited by

15 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 14 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.coling-main.237

@inproceedings{guo2020incorporating,
  title = {{Incorporating Syntax and Frame Semantics in Neural Network for Machine Reading Comprehension}},
  author = {Guo, Shaoru and Guan, Yong and Li, Ru and Li, Xiaoli and Tan, Hongye},
  booktitle = {Proceedings of the 28th International Conference on Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.coling-main.237},
  url = {https://doi.org/10.18653/v1/2020.coling-main.237}
}

References

23 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Collin F. Baker, Charles J. Fillmore, and John B. Lowe. 1998. The berkeley framenet project. In Proceedings of the 17th International Conference on Computational Linguistics, COLING ’98, pages 86–90, Stroudsburg, PA, USA. Association for Computational Linguistics. find our card
  2. Jane Bromley, Isabelle Guyon, Yann LeCun, Eduard Säckinger, and Roopak Shah. 1993. Signature verification using a siamese time delay neural network. In Proceedings of the 6th International Conference on Neural Information Processing Systems, NIPS93, page 737744, San Francisco, CA, USA. Morgan Kaufm… find
  3. Dipanjan Das, Desai Chen, Andr F. T. Martins, Nathan Schneider, and Noah A. Smith. 2014. Frame-semantic parsing. Computational Linguistics, 40(1):9–56. find our card
  4. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805. find
  5. Charles J. Fillmore. 1976. Frame semantics and the nature of language. Annals of the New York Academy of Sciences, 280(1):20–32. find
  6. Shaoru Guo, Ru Li, Hongye Tan, Xiaoli Li, Yong Guan, Hongyan Zhao, and Yueping Zhang. 2020. A frame-based sentence representation for machine reading comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 891–896, Online, July. Association fo… find
  7. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28,… find our card
  8. Meghana Kshirsagar, Sam Thomson, Nathan Schneider, Jaime Carbonell, Noah A. Smith, and Chris Dyer. 2015. Frame-semantic role labeling with heterogeneous annotations. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference… find our card
  9. Chenrui Li, Yuanbin Wu, and Man Lan. 2018. Inference on syntactic and semantic structures for machine comprehension. find
  10. Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In Association for Computational Linguistics (ACL) System Demonstrations, pages 55–60. find our card
  11. Pramod Kaushik Mudrakarta, Ankur Taly, Mukund Sundararajan, and Kedar Dhamdhere. 2018. Did the model understand the question? CoRR, abs/1805.05492. find
  12. Baolin Peng, Zhengdong Lu, Hang Li, and Kam-Fai Wong. 2015. Towards neural network-based reasoning. CoRR, abs/1508.05508. — [arXiv:1508.05508](https://arxiv.org/abs/1508.05508) find
  13. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. find
  14. Matthew Richardson, Christopher J.C. Burges, and Erin Renshaw. 2013. MCTest: A challenge dataset for the open-domain machine comprehension of text. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 193–203, Seattle, Washington, USA, October. Association… find our card
  15. Mrinmaya Sachan and Eric Xing. 2016. Machine comprehension using rich semantic representations. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 486–492, Berlin, Germany, August. Association for Computational Linguistics. find
  16. Kai Sun, Dian Yu, Dong Yu, and Claire Cardie. 2018. Improving machine reading comprehension with general reading strategies. CoRR, abs/1810.13441. find
  17. Adam Trischler, Zheng Ye, Xingdi Yuan, Jing He, Philip Bachman, and Kaheer Suleman. 2016. A parallel-hierarchical model for machine comprehension on sparse data. CoRR, abs/1603.08884. find
  18. Chao Wang and Hui Jiang. 2019. Explicit utilization of general knowledge in machine reading comprehension. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2263–2272, Florence, Italy, July. Association for Computational Linguistics. find
  19. Hai Wang, Mohit Bansal, Kevin Gimpel, and David McAllester. 2015. Machine comprehension with syntax, frames, and semantics. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2:… find
  20. Zhiguo Wang, Wael Hamza, and Radu Florian. 2017. Bilateral multi-perspective matching for natural language sentences. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, pages 4144–4150. find our card
  21. Xiaodong Zhang, Xu Sun, and Houfeng Wang. 2018a. Duplicate question identification by integrating framenet with neural networks. In Thirty-Second AAAI Conference on Artificial Intelligence. find
  22. Zhuosheng Zhang, Yuwei Wu, Zuchao Li, Shexia He, Hai Zhao, Xi Zhou, and Xiang Zhou. 2018b. I know what you want: Semantic learning for text comprehension. CoRR, abs/1809.02794. — [arXiv:1809.02794](https://arxiv.org/abs/1809.02794) find
  23. Zhuosheng Zhang, Yuwei Wu, Hai Zhao, Zuchao Li, Shuailiang Zhang, Xi Zhou, and Xiang Zhou. 2019. Semantics-aware bert for language understanding. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/guo-2020-incorporating-syntax-frame-semantics-neural/