Elamur Research
☰
home / papers / 2018 / A Graph-to-Sequence Model for AMR-to-Text Generation

A Graph-to-Sequence Model for AMR-to-Text Generation

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/P18-1150.pdf

Summary

The paper addresses AMR-to-text generation, which means producing a sentence that expresses the meaning of an Abstract Meaning Representation graph. Earlier neural systems, notably the sequence-to-sequence model of Konstas et al. (2017), first flatten the graph into a bracketed token string. This flattening can place directly linked concepts far apart, and the problem worsens for large graphs. The authors replace the linear encoder with a graph-state LSTM. Each AMR node keeps its own hidden state and memory cell. Over several recurrent steps, each node exchanges information with its neighbours through incoming and outgoing labelled edges, so information spreads gradually across the whole graph. Because all node states are updated at once, encoding can be parallelised. An attention-based LSTM decoder with coverage and a copy mechanism generates the text, and copying handles rare tokens such as names and numbers without hand-written anonymization rules. On the LDC2015E86 benchmark, the model beats a strong sequence-to-sequence baseline and the previous state of the art. This shows that modelling graph structure directly helps generation from semantic graphs.

Key points

  • The model is trained on LDC2015E86 and reaches 23.3 BLEU on the test set. This is 1.3 points higher than the Konstas et al. (2017) system when that system uses identical training data, and 2.3 BLEU above the authors' own seq2seq baseline.
  • With extra Gigaword sentences as training data, the model stays consistently ahead of Konstas et al. (2017) when both use equal amounts of Gigaword data.
  • The encoder uses LSTM gates and a per-node cell. Edge inputs combine label and source-node embeddings, optionally with a character-level LSTM. The authors contrast this gated design with the linear updates of GCN and with the sequential DAG LSTM of Peng et al. (2017).
  • A copy mechanism replaces the rule-based anonymization of dates, numbers and named entities. Example outputs also show G2S recovering structural roles that S2S misses, such as treating "agree" as the subject of "provide".
  • Code and models are released at github.com/freesunshine0316/neural-graph-to-seq-mp.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Machine translation · Natural language processing

Cited by

117 papers of our library cite it (of the 140673 whose bibliographies we read); 14 of them have a page here, 103 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/p18-1150

@inproceedings{song2018graph,
  title = {{A Graph-to-Sequence Model for AMR-to-Text Generation}},
  author = {Song, Linfeng and Zhang, Yue and Wang, Zhiguo and Gildea, Daniel},
  booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2018},
  doi = {10.18653/v1/p18-1150},
  url = {https://doi.org/10.18653/v1/p18-1150}
}

References

33 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations (ICLR). find our card
  2. Laura Banarescu, Claire Bonial, Shu Cai, Madalina Georgescu, Kira Griffitt, Ulf Hermjakob, Kevin Knight, Philipp Koehn, Martha Palmer, and Nathan Schneider. 2013. Abstract meaning representation for sembanking. In Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse, pages 178–186. find our card
  3. Joost Bastings, Ivan Titov, Wilker Aziz, Diego Marcheggiani, and Khalil Simaan. 2017. Graph convolutional encoders for syntax-aware neural machine translation. In Conference on Empirical Methods in Natural Language Processing (EMNLP-17), pages 1957–1967, Copenhagen, Denmark. find our card
  4. Jeffrey Flanigan, Chris Dyer, Noah A. Smith, and Jaime Carbonell. 2016a. CMU at semeval-2016 task 8: Graph-based AMR parsing with infinite ramp loss. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 1202–1206, San Diego, California. find
  5. Jeffrey Flanigan, Chris Dyer, Noah A. Smith, and Jaime Carbonell. 2016b. Generation from abstract meaning representation using tree transducers. In Proceedings of the 2016 Meeting of the North American chapter of the Association for Computational Linguistics (NAACL-16), pages 731–739. find our card
  6. Normunds Gruzitis, Didzis Gosko, and Guntis Barzdins. 2017. RIGOTRIO at SemEval-2017 Task 9: Combining Machine Learning and Grammar Engineering for AMR Parsing and Generation. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 924–928, Vancouver, Canada. find our card
  7. Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O.K. Li. 2016. Incorporating copying mechanism in sequence-to-sequence learning. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL-16), pages 1631–1640, Berlin, Germany. find our card
  8. Caglar Gulcehre, Sungjin Ahn, Ramesh Nallapati, Bowen Zhou, and Yoshua Bengio. 2016. Pointing the unknown words. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL-16), pages 140–149, Berlin, Germany. find
  9. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. ↗
  10. Bevan Jones, Jacob Andreas, Daniel Bauer, Karl Moritz Hermann, and Kevin Knight. 2012. Semantics-based machine translation with hyperedge replacement grammars. In Proceedings of the International Conference on Computational Linguistics (COLING-12), pages 1359–1376. find our card
  11. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. ↗
  12. Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations (ICLR). find our card
  13. Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Proceedings of the 2003 Meeting of the North American chapter of the Association for Computational Linguistics (NAACL-03), pages 48–54. find
  14. Ioannis Konstas, Srinivasan Iyer, Mark Yatskar, Yejin Choi, and Luke Zettlemoyer. 2017. Neural AMR: Sequence-to-sequence models for parsing and generation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL-17), pages 146–157, Vancouver, Canada. find our card
  15. Gerasimos Lampouras and Andreas Vlachos. 2017. Sheffield at semeval-2017 task 9: Transition-based language generation from amr. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 586–591, Vancouver, Canada. find
  16. Xiang Li, Thien Huu Nguyen, Kai Cao, and Ralph Grishman. 2015. Improving event detection with abstract meaning representation. In Proceedings of the First Workshop on Computing News Storylines, pages 11–15, Beijing, China. find
  17. Diego Marcheggiani and Ivan Titov. 2017. Encoding sentences with graph convolutional networks for semantic role labeling. In Conference on Empirical Methods in Natural Language Processing (EMNLP-17), pages 1506–1515, Copenhagen, Denmark. find our card
  18. Ryan McDonald, Koby Crammer, and Fernando Pereira. 2005. Online large-margin training of dependency parsers. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL-05), pages 91–98, Ann Arbor, Michigan. find our card
  19. Simon Mille, Roberto Carlini, Alicia Burga, and Leo Wanner. 2017. Forge at semeval-2017 task 9: Deep sentence generation based on a sequence of graph transducers. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 920–923, Vancouver, Canada. find
  20. Arindam Mitra and Chitta Baral. 2015. Addressing a question answering challenge by combining statistical methods with inductive rule learning and reasoning. In Proceedings of the National Conference on Artificial Intelligence (AAAI-16). find
  21. Kevin P Murphy, Yair Weiss, and Michael I Jordan. 1999. Loopy belief propagation for approximate inference: An empirical study. In Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence, pages 467–475. Morgan Kaufmann Publishers Inc. find
  22. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL-02), pages 311–318. find our card
  23. Nanyun Peng, Hoifung Poon, Chris Quirk, Kristina Toutanova, and Wen-tau Yih. 2017. Cross-sentence n-ary relation extraction with graph LSTMs. Transactions of the Association for Computational Linguistics, 5:101–115. find
  24. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global vectors for word representation. In Conference on Empirical Methods in Natural Language Processing (EMNLP-14), pages 1532–1543. find
  25. Nima Pourdamghani, Kevin Knight, and Ulf Hermjakob. 2016. Generating English from abstract meaning representations. In International Conference on Natural Language Generation (INLG-16), pages 21–25, Edinburgh, UK. find our card
  26. Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL-17), pages 1073–1083, Vancouver, Canada. find
  27. Linfeng Song, Xiaochang Peng, Yue Zhang, Zhiguo Wang, and Daniel Gildea. 2017. AMR-to-text generation with synchronous node replacement grammar. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL-17), pages 7–13, Vancouver, Canada. find our card
  28. Linfeng Song, Zhiguo Wang, Wael Hamza, Yue Zhang, and Daniel Gildea. 2018. Leveraging context information for natural question generation. In Proceedings of the 2018 Meeting of the North American chapter of the Association for Computational Linguistics (NAACL-18), New Orleans. find
  29. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104–3112. find
  30. Kai Sheng Tai, Richard Socher, and Christopher D. Manning. 2015. Improved semantic representations from tree-structured long short-term memory networks. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL-15), pages 1556–1566, Beijing, China. find
  31. Sho Takase, Jun Suzuki, Naoaki Okazaki, Tsutomu Hirao, and Masaaki Nagata. 2016. Neural headline generation on abstract meaning representation. In Conference on Empirical Methods in Natural Language Processing (EMNLP-16), pages 1054–1059, Austin, Texas. find
  32. Aleš Tamchyna, Chris Quirk, and Michel Galley. 2015. A discriminative model for semantics-to-string translation. In Proceedings of the 1st Workshop on Semantics-Driven Statistical Machine Translation (S2MT 2015), pages 30–36, Beijing, China. find
  33. Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. 2016. Modeling coverage for neural machine translation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (ACL-16), pages 76–85, Berlin, Germany. find our card