Elamur Research⌕
☰
home / papers / 2018 / Neural Latent Extractive Document Summarization

Neural Latent Extractive Document Summarization

Paper · 2018 · Xingxing Zhang, Mirella Lapata, Furu Wei et al.

Summary

Extractive summarizers are usually trained on sentence-level True/False labels. Most datasets supply only document–summary pairs, so these labels must be derived heuristically, for example by choosing the sentence subset with the highest ROUGE against the reference, and they may be suboptimal. Zhang, Lapata, Wei and Zhou instead treat each sentence's selection decision as a binary latent variable.

A hierarchical Bi-LSTM extractor (EXTRACT) samples a set of sentences. An attention-based sequence-to-sequence compression model then scores how well those sentences can generate the human summary, combining precision-like and recall-like terms. This score acts as a reward, and the model is optimized with REINFORCE using a learned baseline. Training is initialized from the pre-trained extractor, so the loss comes directly from gold summaries rather than from approximate labels.

On CNN/Dailymail, the resulting LATENT model significantly outperforms both LEAD3 and the strong EXTRACT baseline, and it is competitive with recent extractive and abstractive systems. The work shows that supervising extraction directly with reference summaries is a practical alternative to heuristic labeling.

Key points

  • Data: CNN/Dailymail with See et al. (2017) preprocessing, non-anonymized: 287,226 training, 13,368 validation and 11,490 test pairs. A derived compression corpus has about 1.05M training sentence pairs.
  • Results (ROUGE-1/2/L F1): LATENT scores 41.05/18.77/37.54, compared with EXTRACT at 40.62/18.45/37.14 and LEAD3 at 40.34/17.70/36.57. The differences among these three systems are statistically significant.
  • LATENT achieves the best ROUGE-2 in the comparison, beating SummaRuNNer, EXTRACT-CNN, REFRESH, pointer+coverage and the RL-based abstractive models. abstract-RL still leads on ROUGE-1 (41.16) and ROUGE-L (39.08).
  • The reward is a weighted mix of precision and recall terms with α = 0.5. Setup: FastText 300-d embeddings, 300-unit one-layer LSTMs, top-3 sentences selected at inference.
  • Rewriting LATENT's output with the sentence compression model (LATENT+COMPRESS) lowers scores sharply to 36.69/15.43/34.33. The likely cause is that it drops phrases that matter at document level.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of EdinburghEdinburgh · United Kingdom255 works · 68,771 citations🏢Microsoft Research AsiaBeijing · China54 works · 59,921 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Summarization · 159 works

Cite

Cite as Xingxing Zhang, Mirella Lapata, Furu Wei and Ming Zhou. 2018. Neural Latent Extractive Document Summarization. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/d18-1088

@inproceedings{zhang2018neural,
  title = {{Neural Latent Extractive Document Summarization}},
  author = {Xingxing Zhang, Mirella Lapata, Furu Wei and Ming Zhou},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1088},
  url = {https://doi.org/10.18653/v1/d18-1088}
}

References

6 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In In Proceedings of the 3rd International Conference on Learning Representations, San Diego, California.
  2. Jaime Carbonell and Jade Goldstein. 1998. The use of mmr, diversity-based reranking for reordering documents and producing summaries. In Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, pages 335–336. ACM. find
  3. Asli Celikyilmaz, Antoine Bosselut, Xiaodong He, and Yejin Choi. 2018. Deep communicating agents for abstractive summarization. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), p…
  4. Jianpeng Cheng and Mirella Lapata. 2016. Neural summarization by extracting sentences and words. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 484–494, Berlin, Germany.
  5. John M Conroy and Dianne P O’leary. 2001. Text summarization via hidden markov models. In Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, pages 406–407. ACM. find
  6. Elena Filatova and Vasileios Hatzivassiloglou. 2004. Event-based extractive summarization. In Text Summarization Branches Out: Proceedings of the ACL-04 Workshop, pages 104–111, Barcelona, Spain. \#\# Conclusions Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustaf… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2018-neural-latent-extractive-document-summarization/