Elamur Research⌕
☰
home / papers / 2020 / Towards Faithful Neural Table-to-Text Generation with Content-Matching Constraints

Towards Faithful Neural Table-to-Text Generation with Content-Matching Constraints

Paper · 2020 · Zhenyi Wang, Xiaoyang Wang, Bang An et al.

Summary

Neural table-to-text systems often produce descriptions that are unfaithful to their input. They add facts absent from the table (hallucinations) or leave out facts that are present. This happens partly because reference texts in training data are themselves noisy. The authors address this for multi-sentence generation from knowledge-base tables. They replace the usual RNN Seq2Seq models with a Transformer encoder-decoder that takes the table as a linearized sequence. Two auxiliary training signals push the output to stay close to the input. One aligns the mean embedding of the table with the mean embedding of the reference text. The other uses an optimal-transport distance to match nouns from the table against nouns in the generated text. To measure faithfulness automatically, they introduce PARENT-T, a table-focused variant of PARENT. On the Wikiperson knowledge-base dataset, automatic metrics and blind human ratings both favour the model over prior state-of-the-art systems. The excerpt provided omits the exact scores. The work matters because it treats fidelity to the source data as an explicit training objective rather than a side effect.

Key points

  • The training loss has three parts: maximum-likelihood loss, a table–text "disagreement" loss (L2 distance between mean embeddings), and an optimal-transport (OT) loss over key words.
  • The OT loss uses cosine distance between word embeddings and is approximated with IPOT. Key words are nouns extracted via NLTK, which also cuts computation compared with matching every word. This differentiable approach avoids the high variance of policy-gradient methods.
  • Tables are flattened into sequences, with slot types wrapped in "<" and ">" tokens. Adding graph neural network features did not help in preliminary tests.
  • PARENT-T is a new automatic metric that scores content overlap between generated text and the table alone, building on PARENT (Dhingra et al., 2019).
  • Experiments use the Wikiperson dataset (Wang et al., 2018), which has multi-sentence targets, unlike single-sentence WikiBio. Human evaluation covered 100 randomly sampled test outputs per model, rated blind by trained raters.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University at Buffalo, State University of New YorkBuffalo · United States12 works · 318 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 1Research subjects and languages the work is about.Text generation · 202 works

Cite

Cite as Zhenyi Wang, Xiaoyang Wang, Bang An, Dong Yu and Changyou Chen. 2020. Towards Faithful Neural Table-to-Text Generation with Content-Matching Constraints. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.acl-main.101

@inproceedings{wang2020faithful,
  title = {{Towards Faithful Neural Table-to-Text Generation with Content-Matching Constraints}},
  author = {Zhenyi Wang, Xiaoyang Wang, Bang An, Dong Yu and Changyou Chen},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.101},
  url = {https://doi.org/10.18653/v1/2020.acl-main.101}
}

References

31 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the International Conference on Learning Representations.
  2. Regina Barzilay and Mirella Lapata. 2005. Collective content selection for concept-to-text generation. In HLT/EMNLP 2005, Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference, 6-8 October 2005, Vancouver, British Colum…
  3. Bhuwan Dhingra, Manaal Faruqui, Ankur Parikh, Ming-Wei Chang, Dipanjan Das, and William W. Cohen. 2019. Handling divergent reference texts when evaluating table-to-text generation. In Proceedings of 57th Annual Meeting of the Association for Computational Linguistics.
  4. Sebastian Gehrmann, Falcon Z. Dai, Henry Elder, and Alexander M. Rush. 2018. End-to-end content and plan selection for data-to-text generation. In Proceedings of International Conference on Natural Language Generation. find
  5. Li Gong, Josep Crego, and Jean Senellart. 2019. Enhanced transformer model for data-to-text generation. In Proceedings of the 3rd Workshop on Neural Generation and Translation (EMNLP). find
  6. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the International Conference for Learning Representations.
  7. Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M. Rush. 2017. Open-NMT: Open-source toolkit for neural machine translation. In Proceedings of Annual Meeting of the Association for Computational Linguistics. find
  8. Harold W. Kuhn. 1955. The hungarian method for the assignment problem. In Naval research logistics quarterly. find
  9. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. Albert: A lite bert for self-supervised learning of language representations. In https://arxiv.org/abs/1909.11942. ↗
  10. Rémi Lebret, David Grangier, and Michael Auli. 2016a. Neural text generation from structured data with application to the biography domain. In Proceedings of the Conference on Empirical Methods in Natural Language Processing.
  11. Rémi Lebret, David Grangier, and Michael Auli. 2016b. Neural text generation from structured data with application to the biography domain. In Proceedings of the Conference on Empirical Methods in Natural Language Processing.
  12. Percy Liang, Michael I. Jordan, and Dan Klein. 2009. Learning semantic correspondences with less supervision. In Proceedings of the Annual Meeting of the Association for Computational Linguistics and International Joint Conference on Natural Language Processing of the AFNLP.
  13. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Proceedings of Text Summarization Branches Out.
  14. Tianyu Liu, Fuli Luo, Qiaolin Xia, Shuming Ma, Baobao Chang, and Zhifang Sui. 2019a. Hierarchical encoder with auxiliary supervision for neural table-to-text generation: Learning better representation for tables. In Proceedings of the AAAI Conference on Artificial Intelligence. find
  15. Tianyu Liu, Fuli Luo, Pengcheng Yang, Wei Wu, Baobao Chang, and Zhifang Sui. 2019b. Towards comprehensive description generation from factual attribute-value tables. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. find
  16. Tianyu Liu, Kexiang Wang, Lei Sha, Baobao Chang, and Zhifang Sui. 2018. Table-to-text generation by structure-aware seq2seq learning. In Proceedings of the AAAI Conference on Artificial Intelligence. find
  17. Edward Loper and Steven Bird. Nltk: The natural language toolkit. https://arxiv.org/abs/cs/0205028.2002. In Shuming Ma, Pengcheng Yang, Tianyu Liu, Peng Li, Jie Zhou, and Xu Sun. 2019. Key fact as pivot: A two-stage model for low resource table-to-text generation. In https://arxiv.org/pdf/1908.03067… ↗
  18. Feng Nie, Jinpeng Wang, Jin-Ge Yao, Rong Pan, and Chin-Yew Lin. 2018. Operation-guided neural networks for high fidelity data-to-text generation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. find
  19. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics.
  20. Laura Perez-Beltrachini and Mirella Lapata. 2018. Bootstrapping generators from noisy data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. find
  21. Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In https://arxiv.org/abs/1704.04368. ↗
  22. Linfeng Song, Yue Zhang, Zhiguo Wang, and Daniel Gildea. 2018. A graph-to-sequence model for amr-to-text generation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics.
  23. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems.
  24. Ran Tian, Shashi Narayan, Thibault Sellam, and Ankur P. Parikh. 2019. Sticking to the facts: Confident decoding for faithful data-to-text generation. In https://arxiv.org/abs/1910.08684. — [arXiv:1910.08684](https://arxiv.org/abs/1910.08684) ↗
  25. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems.
  26. Qingyun Wang, Xiaoman Pan, Lifu Huang, Boliang Zhang, Zhiying Jiang, Heng Ji, and Kevin Knight. 2018. Describing a knowledge base. In International Conference on Natural Language Generation. find
  27. Rui Wang, Andrew Finch, Masao Utiyama, and Eiichiro Sumita. 2017. Sentence embedding for neural machine translation domain adaptation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. find
  28. Ronald J. Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. In Machine Learning. find
  29. Sam Wiseman, Stuart M. Shieber, and Alexander M. Rush. 2018. Learning neural templates for text generation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing.
  30. Yujia Xie, Xiangfeng Wang, Ruijia Wang, and Hongyuan Zha. 2018. A fast proximal point method for computing exact wasserstein distance. In https://arxiv.org/pdf/1802.04307.pdf. — [arXiv:1802.04307](https://arxiv.org/abs/1802.04307) ↗
  31. Mingming Yang, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita, Min Zhang, and Tiejun Zhao. 2019. Sentence-level agreement for neural machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2020-faithful-neural-table-text-generation/