Elamur Research⌕
☰
home / papers / 2016 / What to talk about and how? Selective Generation using LSTMs with Coarse-to-Fine Alignment

What to talk about and how? Selective Generation using LSTMs with Coarse-to-Fine Alignment

Summary

The paper addresses selective generation: turning a database with many redundant event records into a short natural-language text. This requires deciding which records matter (content selection) and how to phrase them (surface realization). Earlier systems usually handled these steps separately and depended on domain-specific features, parsers, templates or externally estimated alignments. The authors instead train one neural encoder-aligner-decoder end to end on record–text pairs, with no ground-truth selection labels. A bidirectional LSTM encodes all records. A new coarse-to-fine aligner then narrows the choice of records in two stages: a pre-selector followed by a refiner, both working over low- and high-level views of the input. Finally, an LSTM decoder writes the description word by word while aligning each word to a record. On WeatherGov, the model set new state-of-the-art results for both selection and generation, and a k-nearest-neighbor beam filter improved it slightly further. On the small RoboCup dataset, it matched or beat prior systems. This shows that a feature-free neural approach can move to a new domain without hand engineering.

Key points

  • Data: WeatherGov has 29,528 scenarios, each with 36 weather records and a forecast averaging 28.7 words. RoboCup has only 1,539 pairs of soccer events and commentary (about 1,000 used for training), averaging 2.4 records and 5.7 words per scenario.
  • WeatherGov results: relative gains of 12% in selection F-1 and 59% in BLEU over the previous best. The k-NN beam filter (M = 2, K = 1) raised sBLEU from 61.01 to 61.76 and cBLEU from 70.39 to 71.23.
  • Ablations (dev set):
    • The coarse-to-fine aligner reached F-1 76.28, sBLEU 65.58 and cBLEU 75.78. The standard Bahdanau-style aligner reached 60.35, 63.54 and 74.90.
    • Removing the LSTM encoder lowered F-1 to 57.45.
  • RoboCup results: using an ensemble of five models, the system reached selection F-1 81.58 (best reported) and sBLEU 25.28, compared with 24.88 for Konstas and Lapata (2012). With ground-truth selection given, it scored sBLEU_G 29.40, competitive with the best prior result of 30.90.
  • Embeddings: the learned word embeddings group related words, such as gusts–gust and isolated–scattered. Initializing or concatenating with pre-trained word2vec vectors brought only negligible improvement.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

112 papers of our library cite it (of the 140884 whose bibliographies we read); 10 of them have a page here, 102 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n16-1086

@inproceedings{mei2016what,
  title = {{What to talk about and how? Selective Generation using LSTMs with Coarse-to-Fine Alignment}},
  author = {Mei, Hongyuan and Bansal, Mohit and Walter, Matthew R.},
  booktitle = {Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year = {2016},
  doi = {10.18653/v1/n16-1086},
  url = {https://doi.org/10.18653/v1/n16-1086}
}

References

33 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Gabor Angeli, Percy Liang, and Dan Klein. 2010. A simple domain-independent probabilistic approach to generation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 502–512. find our card
  2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. find our card
  3. Regina Barzilay and Mirella Lapata. 2005. Collective content selection for concept-to-text generation. In Proceedings of the Human Language Technology Conference and the Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pages 331–338. find
  4. Regina Barzilay and Lillian Lee. 2004. Catching the drift: Probabilistic content models, with applications to generation and summarization. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics Human Language Technologies (NAACL HLT), pages 1… find our card
  5. Anja Belz. 2008. Automatic generation of weather forecast texts using comprehensive probabilistic generation-space models. Natural Language Engineering, 14(04):431–455. James Bergstra, Olivier Breuleux, Frédéric Bastien, Pascal Lamblin, Razvan Pascanu, Guillaume Desjardins, Joseph Turian, David Wa… find
  6. David L. Chen and Raymond J. Mooney. 2008. Learning to sportscast: a test of grounded language acquisition. In Proceedings of the International Conference on Machine Learning (ICML), pages 128–135. find
  7. David L. Chen, Joohyun Kim, and Raymond J. Mooney. 2010. Training a multilingual sportscaster: Using perceptual context to learn language. Journal of Artificial Intelligence Research, 37:397–435. find
  8. Alex Graves, Mohamed Abdel-rahman, and Geoffrey Hinton. 2013. Speech recognition with deep recurrent neural networks. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6645–6649. find
  9. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735– 1780. find
  10. Andrej Karpathy and Li Fei-Fei. 2015. Deep visual-semantic alignments for generating image descriptions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3128–3137. find our card
  11. Joohyun Kim and Raymond J Mooney. 2010. Generative alignment and semantic parsing for learning from ambiguous supervision. In Proceedings of the International Conference on Computational Linguistics (COLING), pages 543–551. find
  12. Diederik Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations (ICLR). find
  13. Ioannis Konstas and Mirella Lapata. 2012. Unsupervised concept-to-text generation with hypergraphs. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics Human Language Technologies (NAACL HLT), pages 752–761. find
  14. Ioannis Konstas and Mirella Lapata. 2013. Inducing document plans for concept-to-text generation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1503– 1514. find
  15. Percy Liang, Michael I. Jordan, and Dan Klein. 2009. Learning semantic correspondences with less supervision. In Proceedings of the Joint Conference of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (ACL/IJCNL… find our card
  16. Wei Lu and Hwee Tou Ng. 2011. A probabilistic forest-to-string model for language generation from typed lambda calculus expressions. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1611–1622. find
  17. Wei Lu, Hwee Tou Ng, Wee Sun Lee, and Luke S Zettlemoyer. 2008. A generative model for parsing natural language to meaning representations. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 783–792. find
  18. Wei Lu, Hwee Tou Ng, and Wee Sun Lee. 2009. Natural language generation with tree conditional random fields. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 400–409. find
  19. Hongyuan Mei, Mohit Bansal, and Matthew R. Walter. 2015. Listen, attend, and walk: Neural mapping of navigational instructions to action sequences. arXiv preprint arXiv:1506.04089. find
  20. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In Proceedings of the International Conference on Learning Representations (ICLR). find
  21. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2001. BLEU: a method for automatic evaluation of machine translation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 311–318. find our card
  22. Razvan Pascanu, Caglar Gulcehre, Kyunghyun Cho, and Yoshua Bengio. 2014. How to construct deep recurrent neural networks. In Proceedings of the International Conference on Learning Representations (ICLR). find
  23. Ehud Reiter, Robert Dale, and Zhiwei Feng. 2000. Building natural language generation systems, volume 33. find
  24. MIT Press. Radu Soricut and Daniel Marcu. 2006. Stochastic language generation using WIDL-expressions and its application in machine translation and summarization. In Proceedings of the International Conference on Computational Linguistics and the Annual Meeting of the Association for Computational… find
  25. Amanda Stent, Rashmi Prasad, and Marilyn Walker. 2004. Trainable sentence planning for complex information presentation in spoken dialog systems. In Proceedings of the 42nd annual meeting on association for computational linguistics, page 79. Association for Computational Linguistics. find
  26. Ilya Sutskever, Oriol Vinyals, and Quoc V. Lee. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems (NIPS). find
  27. Oriol Vinyals, Lukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey Hinton. 2014. Grammar as a foreign language. arXiv preprint arXiv:1412.7449. find
  28. Oriol Vinyals, Samy Bengio, and Manjunath Kudlur. 2015a. Order matters: Sequence to sequence for sets. arXiv preprint arXiv:1511.06391. find
  29. Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2015b. Show and tell: A neural image caption generator. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 3156–3164. find our card
  30. Marilyn A Walker, Owen Rambow, and Monica Rogati. 2001. Spot: A trainable sentence planner. In Proceedings of the second meeting of the North American Chapter of the Association for Computational Linguistics on Language technologies, pages 1–8. Association for Computational Linguistics. find
  31. Yuk Wah Wong and Raymond J Mooney. 2007. Generation by inverting a semantic parser that uses statistical machine translation. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics Human Language Technologies (NAACL HLT), pages 172–179. find our card
  32. Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard Zemel, and Yoshua Bengio. 2015. Show, attend and tell: Neural image caption generation with visual attention. In Proceedings of the International Conference on Machine Learning (ICML). find our card
  33. Wojciech Zaremba, Ilya Sutskever, and Oriol Vinyals. 2014. Recurrent neural network regularization. arXiv preprint arXiv:1409.2329. find our card

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mei-2016-what-talk-about-how-selective/