Elamur Research⌕
☰
home / papers / 2017 / TL;DR: Mining Reddit to Learn Automatic Summarization

TL;DR: Mining Reddit to Learn Automatic Summarization

Summary

Neural abstractive summarizers need very large training sets, and nearly all large English corpora for this task come from news. Völske et al. address this lack of genre variety by turning to social media. Reddit authors often add a "TL;DR" line to long posts, so each such post contains a text and a summary by the same writer. The authors processed a public Reddit dump covering 2006–2016. They kept posts with plausible TL;DR spellings, removed bot accounts, and split each remaining post at the marker into content and summary, with simple sanity rules. The result is Webis-TLDR-17, about 4 million content-summary pairs. This matches English Gigaword in size but consists of informal, everyday writing. In a manual check of a random sample, 95% of pairs were correct. The paper also offers heuristic subsets: likely genuine summaries, question-style TL;DRs, and vulgar summaries. Early runs of existing neural models on the data gave encouraging output, judged only by inspection. The method should also work on general web crawls.

Key points

  • Source data: 286 million submissions and about 1.6 billion comments from Reddit, taken from the pushshift.io crawl.
  • Filtering pipeline:
    • A loose pattern (tl.{0,3}dr) was used first.
    • The 100 most frequent spellings were reviewed by hand, and 33 reliable variants were kept.
    • Bot accounts were removed.
    • Posts with multiple TL;DRs were discarded, and each summary had to be shorter than its content.
  • Final corpus: about 1.67 million submission pairs and 2.38 million comment pairs. A subset with at least 100 words of content is also provided.
  • Verticals:
    • 966,430 pairs share noun phrases between content and summary, a lower bound on true summaries.
    • 78,710 pairs have question summaries.
    • 299,145 pairs have vulgar summaries, flagged with a list of over 500 offensive words.
  • The corpus was used to train the models of Nallapati et al. (2016) and Rush et al. (2015), while quantitative evaluation was deferred to future work. CommonCrawl and Clueweb12 contain over 2 million pages with the TL;DR pattern.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

109 papers of our library cite it (of the 140884 whose bibliographies we read); 5 of them have a page here, 104 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/w17-4508

@inproceedings{volske2017tl,
  title = {{TL;DR: Mining Reddit to Learn Automatic Summarization}},
  author = {Völske, Michael and Potthast, Martin and Syed, Shahbaz and Stein, Benno},
  booktitle = {Proceedings of the Workshop on New Frontiers in Summarization},
  year = {2017},
  doi = {10.18653/v1/w17-4508},
  url = {https://doi.org/10.18653/v1/w17-4508}
}

References

10 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015. pages 379–389. http://aclweb.org/a… ↗
  2. Rasim M. Alguliev, Ramiz M. Aliguliyev, and Nijat R. Isazade. 2013. Multiple documents summarization based on evolutionary optimization algorithm. Expert Syst. Appl. 40(5):1675–1689. https://doi.org/10.1016/j.eswa.2012.09.014. — [doi:10.1016/j.eswa.2012.09.014](https://doi.org/10.1016/j.eswa.2012.09.014) ↗
  3. Sumit Chopra, Michael Auli, and Alexander M Rush. 2016. Abstractive Sentence Summarization with Attentive Recurrent Neural Networks. In NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego Californi… find our card
  4. Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. CoRR abs/1704.04368. To appear in ACL’17. http://arxiv.org/abs/1704.04368. ↗
  5. Mahak Gambhir and Vishal Gupta. 2017. Recent automatic text summarization techniques: a survey. Artificial Intelligence Review 47(1):1–66. https://doi.org/10.1007/s10462-016-9475-9. — [doi:10.1007/s10462-016-9475-9](https://doi.org/10.1007/s10462-016-9475-9) ↗
  6. Karl Moritz Hermann, Tomás Kociský, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December… find our card
  7. Baotian Hu, Qingcai Chen, and Fangze Zhu. 2015. LCSTS: A Large Scale Chinese Short Text Summarization Dataset. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015. Association for Computational Linguistics, pa… find
  8. Ju-Hong Lee, Sun Park, Chan-Min Ahn, and Daeho Kim. 2009. Automatic generic document summarization based on non-negative matrix factorization. Inf. Process. Manage. 45(1):20–34. https://doi.org/10.1016/j.ipm.2008.06.002. — [doi:10.1016/j.ipm.2008.06.002](https://doi.org/10.1016/j.ipm.2008.06.002) ↗
  9. Ramesh Nallapati, Bowen Zhou, Cı́cero Nogueira dos Santos, Çaglar Gülçehre, and Bing Xiang. 2016. Abstractive text summarization using sequence-to-sequence rnns and beyond. In Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, CoNLL 2016, Berlin, Germany, August… find
  10. You Ouyang, Wenjie Li, Sujian Li, and Qin Lu. 2011. Applying regression models to query-focused multi-document summarization. Inf. Process. Manage. 47(2):227–237. https://doi.org/10.1016/j.ipm.2010.03.005. — [doi:10.1016/j.ipm.2010.03.005](https://doi.org/10.1016/j.ipm.2010.03.005) ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/volske-2017-tl-dr-mining-reddit-learn/