Elamur Research⌕
☰
home / papers / 2026 / ThreadSumm: Summarization of Nested Discourse Threads Using Tree of Thoughts

ThreadSumm: Summarization of Nested Discourse Threads Using Tree of Thoughts

Paper · 2026 · Olubusayo Olabisi, Ekata Mitra and Ameeta Agrawal

Summary

Forum discussions form tree-like threads where replies, quotes and side topics interleave. Standard LLM summarizers often flatten this structure and drop minority viewpoints or lose logical flow. The authors propose ThreadSumm, a staged LLM pipeline that treats thread summarization as structured reasoning.

The pipeline has five steps:

  1. Few-shot prompting extracts discourse aspects (who/what/where).
  2. Atomic Content Units (ACUs) are generated for each aspect.
  3. A zero-shot prompt reorders the ACUs into a logical sequence without relying on timestamps or other thread metadata.
  4. Several candidate paragraphs are written from the ordered units.
  5. A Tree of Thoughts search scores the candidates for coherence and coverage, using the LLM as judge, and keeps the best one.

Experiments use three LLMs (GPT4, Claude, LLaMA) on Reddit and Stack threads, plus a single Bitcoin forum thread used as a visualization case study. ThreadSumm is reported to beat vanilla prompting and the mReddit baseline, retaining more aspects and opinions. This matters for any setting where multi-party, non-linear conversations must be condensed faithfully.

Key points

  • Pipeline stages: The system first pulls out discourse aspects, then builds ACUs, arranges those units in a logical order, drafts candidate paragraphs, and finally evaluates and selects among them with Tree of Thoughts. The selection criterion is maximum coherence and coverage.
  • Datasets:
    • Reddit: 250 instances, about 641 input tokens, about 7.88 docs per example.
    • Stack (community Q&A): 117 instances, about 1207 input tokens, about 9.72 docs per example.
    • Bitcoin: one instance with 26 docs and no reference summary.
  • Ablation: The full system reaches QAGS 85.71 and SummaC 63.72. Removing any one of aspect extraction, sentence reordering, or ToT drops QAGS to 57.14. SummaC falls to 50.19, 45.06, and 44.44 respectively.
  • Default search setting: 1 reordering proposal (r = 1) and 2 paragraph proposals (p = 2). Varying r and p gave mixed, model-dependent changes.
  • Limitations:
    • Only English text threads were used.
    • The same LLMs both generate and score summaries.
    • The LLM-based scores were not calibrated against human judgments.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Portland State UniversityPortland · United States6 works · 53 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Discourse analysis · 466 worksSummarization · 232 works

Cite

Cite as Olubusayo Olabisi, Ekata Mitra and Ameeta Agrawal. 2026. ThreadSumm: Summarization of Nested Discourse Threads Using Tree of Thoughts. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1486

@inproceedings{olabisi2026threadsumm,
  title = {{ThreadSumm: Summarization of Nested Discourse Threads Using Tree of Thoughts}},
  author = {Olubusayo Olabisi, Ekata Mitra and Ameeta Agrawal},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1486},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1486}
}

References

29 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Forrest Bao, Ruixuan Tu, Ge Luo, Yinfei Yang, Hebi Li, Minghui Qiu, Youbiao He, and Cen Chen. 2023. Do-casref: An empirical study on repurposing reference-based summary quality metrics as reference-free metrics. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1226–123… find
  2. Yapei Chang, Kyle Lo, Tanya Goyal, and Mohit Iyyer. 2023. Booookscore: A systematic exploration of book-length summarization in the era of llms. arXiv preprint arXiv:2310.00785. find
  3. Alexander Richard Fabbri, Faiaz Rahman, Imad Rizvi, Borui Wang, Haoran Li, Yashar Mehdad, and Dragomir Radev. 2021. Convosumm: Conversation summarization benchmark and improved abstractive summarization with argument mining. In Proceedings of the 59th Annual Meeting of the Association for Computatio… find
  4. Alexander Richard Fabbri, Xiaojian Wu, Srini Iyer, Haoran Li, and Mona Diab. 2022. Answersumm: A manually-curated dataset and pipeline for answer summarization. In Proceedings of the 2022 conference of the north american chapter of the association for computational linguistics: human language techno… find
  5. Melika Golestani, Seyedeh Zahra Razavi, Zeinab Borhanifard, Farnaz Tahmasebian, and Hesham Faili. 2021. Using bert encoding and sentence-level language model for sentence ordering. In International Conference on Text, Speech, and Dialogue, pages 318–330. Springer. find
  6. Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. 2024. Large language model based multi-agents: A survey of progress and challenges. arXiv preprint arXiv:2402.01680.
  7. Xiaobo Guo and Soroush Vosoughi. 2024. Modabs: Multi-objective learning for dynamic aspect-based summarization. arXiv preprint arXiv:2406.03479. find
  8. Doris Hoogeveen, Karin M Verspoor, and Timothy Baldwin. 2015. Cqadupstack: A benchmark data set for community question-answering research. In Proceedings of the 20th Australasian document computing symposium, pages 1–8. find
  9. Tatsuya Ishigaki, Hidetaka Kamigaito, Hiroya Takamura, and Manabu Okumura. 2019. Discourse-aware hierarchical attention network for extractive single-document summarization. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP), pages 497–506, Varna… find
  10. Hyuntak Kim and Byung-Hak Kim. 2025. NexusSum: Hierarchical LLM agents for long-form narrative summarization. arXiv preprint arXiv:2505.24575. Accepted to the main track of ACL 2025. find
  11. Jingun Kwon, Naoki Kobayashi, Hiroya Takamura, and Manabu Okumura. 2021. Considering nested tree structure in sentence extractive summarization with pre-trained transformer. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4018–4030, Online and… find
  12. Philippe Laban, Tobias Schnabel, Paul N. Bennett, and Marti A. Hearst. 2022. Summac: Re-visiting nli-based models for inconsistency detection in summarization. Transactions of the Association for Computational Linguistics, 10:163–177. find
  13. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81.
  14. Yijun Liu, Frederick Choi, and Eshwar Chandrasekharan. 2025. Needling through the threads: A visualization tool for navigating threaded online discussions. arXiv preprint arXiv:2506.11276. — [arXiv:2506.11276](https://arxiv.org/abs/2506.11276) find
  15. Yixin Liu, Alexander R Fabbri, Pengfei Liu, Yilun Zhao, Linyong Nan, Ruilin Han, Simeng Han, Shafiq Joty, Chien-Sheng Wu, Caiming Xiong, et al. 2022. Revisiting the gold standard: Grounding summarization evaluation with robust human evaluation. arXiv preprint arXiv:2212.07981.
  16. Lajanugen Logeswaran, Honglak Lee, and Dragomir Radev. 2016. Sentence ordering using recurrent neural networks. find
  17. Jing Ma. 2025. Input order shapes llm semantic alignment in multi-document summarization. arXiv preprint arXiv:2512.02665. — [arXiv:2512.02665](https://arxiv.org/abs/2512.02665) find
  18. Kawin Mayilvaghanan, Siddhant Gupta, and Ayush Kumar. 2025. Spot the blindspots: Systematic identification and quantification of fine-grained llm biases in contact center call summarization. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, pa… find
  19. Olubusayo Olabisi and Ameeta Agrawal. 2024. Understanding position bias effects on fairness in social multi-document summarization. arXiv preprint arXiv:2405.01790. find
  20. Olubusayo Olabisi, Aaron Hudson, Antonie Jetter, and Ameeta Agrawal. 2022. Analyzing the dialect diversity in multi-document summaries. In Proceedings of the 29th International Conference on Computational Linguistics, pages 6208–6221. find
  21. Keighley Overbay, Jaewoo Ahn, Fatemeh Pesaran Zadeh, Joonsuk Park, and Gunhee Kim. 2023. mred-ditsum: A multimodal abstractive summarization dataset of reddit threads with images. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4117–4132. find
  22. Xu Sun, Lionel Delphin-Poulat, Christèle Tarnec, and Anastasia Shimorina. 2025. Posum-bench: Benchmarking position bias in llm-based conversational summarization. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 7996–8020. find
  23. Suzan Verberne, Emiel Krahmer, Sander Wubben, and Antal van den Bosch. 2020. Query-based summarization of discussion threads. Natural Language Engineering, 26(1):3–29. find
  24. Alex Wang, Kyunghyun Cho, and Mike Lewis. 2020. Asking and answering questions to evaluate the factual consistency of summaries. arXiv preprint arXiv:2004.04228. find
  25. Weiqi Wu, Shen Huang, Yong Jiang, Pengjun Xie, Fei Huang, and Hai Zhao. 2025. Unfolding the headline: Iterative self-questioning for news retrieval and timeline summarization. arXiv preprint arXiv:2501.00888. find
  26. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809–11822. find
  27. Amy Zhang, Bryan Culbertson, and Praveen Paritosh. 2017. Characterizing online discussion using coarse discourse sequences. In proceedings of the international AAAI conference on web and social media, volume 11, pages 357–366. find
  28. Shiyue Zhang, Asli Celikyilmaz, Jianfeng Gao, and Mohit Bansal. 2021. Emailsum: Abstractive email thread summarization. arXiv preprint arXiv:2107.14691. find
  29. Yusen Zhang, Ruoxi Sun, Yanfei Chen, Tomas Pfister, Rui Zhang, and Sercan Arik. 2024. Chain of agents: Large language models collaborating on long-context tasks. Advances in Neural Information Processing Systems, 37:132208–132237. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/olabisi-2026-threadsumm-summarization-nested-discourse-threads/