Elamur Research⌕
☰
home / papers / 2026 / Not Worth Mentioning? A Pilot Study on Salient Proposition Annotation

Not Worth Mentioning? A Pilot Study on Salient Proposition Annotation

Paper · 2026 · Amir Zeldes, Katherine Conhaim and Lauren Levine

Summary

Extractive summarization assumes that some propositions in a text matter more than others. Yet there has been little effort to define proposition salience in a measurable, graded way for real data. The authors carry over a summary-based method from Salient Entity Extraction (SEE): a unit's salience is the number of independent summaries that mention it. They score propositions this way on the GUM corpus test set, which provides five summaries per document. Pre-existing eRST Elementary Discourse Units (EDUs) serve as the propositions, so every word belongs to exactly one unit. Annotators decided, for each summary, whether each EDU matches its content exactly or only approximately triggers it. They also marked component (aggregate) alignments and duplicates. Inter-annotator agreement was moderate under strict comparison and rose when interchangeable duplicates were tolerated. A mixed-effects beta-binomial model, which controls for position, length, sentencehood and relation type, found RST centrality to be the strongest predictor of salience. Overall model fit remained weak, which shows that salient propositions are still poorly understood. The work provides an initial graded resource and a reusable annotation tool.

Key points

  • Data: GUM V12 test set with 32 documents across 16 spoken and written English genres. It contains 30,255 tokens and 3,779 EDUs, which yield 18,895 proposition–summary alignments.
  • Sub-sentential alignment is common: about 78.5% of sentences never appear in summaries, 10.7% appear in full, and 10.8% appear only in part.
  • Agreement:
    • Tolerating duplicates raises micro κ by 5.9 points.
    • Component alignments cause the most disagreement (+8.6 when ignored), compared with +3.98 for other approximate alignments.
  • Salience and discourse structure:
    • Salience correlates with RST centrality (reported r² = −0.287, p < 0.0001) and with earlier position in the document.
    • In the regression, centrality ranks first by AIC, followed by relation type and then length.
    • The model fits weakly (marginal r² 0.131, conditional r² 0.159). Its binary accuracy is 73.87%, against a 72.33% majority baseline.
  • Resources and plans:
    • The GlowLyter annotation interface is released under Apache 2.0, along with public guidelines, and the data will ship with GUM.
    • The authors plan to extend annotation to the rest of GUM and to GENTLE.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Georgetown UniversityWashington · United States88 works · 4,732 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Amir Zeldes, Katherine Conhaim and Lauren Levine. 2026. Not Worth Mentioning? A Pilot Study on Salient Proposition Annotation. Proceedings of the 20th Linguistic Annotation Workshop (LAW XX).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.law-main.14

@inproceedings{zeldes2026not,
  title = {{Not Worth Mentioning? A Pilot Study on Salient Proposition Annotation}},
  author = {Amir Zeldes, Katherine Conhaim and Lauren Levine},
  booktitle = {Proceedings of the 20th Linguistic Annotation Workshop (LAW XX)},
  year = {2026},
  doi = {10.18653/v1/2026.law-main.14},
  url = {https://doi.org/10.18653/v1/2026.law-main.14}
}

References

27 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tatsuya Aoyama, Shabnam Behzad, Luke Gessler, Lauren Levine, Jessica Lin, Yang Janet Liu, Siyao Peng, Yilun Zhu, and Amir Zeldes. 2023. GENTLE: A genre-diverse multilayer challenge set for English NLP and linguistic evaluation. In Proceedings of the 17th Linguistic Annotation Workshop (LAW-XVII), pa…
  2. Kuan-Yu Chen, Shih-Hung Liu, Berlin Chen, and Hsin-Min Wang. 2018. An information distillation framework for extractive summarization. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 26(1):161–170. find
  3. Jianpeng Cheng and Mirella Lapata. 2016. Neural summarization by extracting sentences and words. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 484–494, Berlin, Germany. Association for Computational Linguistics.
  4. Milan Dojchinovski, Dinesh Reddy, Tomáš Kliegr, Tomáš Vitvar, and Harald Sack. 2016. Crowdsourced corpus with entity salience annotations. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 3307–3311, Portorož, Slovenia. European Language Resou…
  5. Jesse Dunietz and Daniel Gillick. 2014. A new entity salience task with millions of training examples. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, volume 2: Short Papers, pages 205–209, Gothenburg, Sweden. Association for Computatio…
  6. Pulkit Dwivedi, Preet Kamal, Mansi Kajal, Nikita Rattan, and Namit Chawla. 2025. Nyx: An extractive text summarization framework using TF-ISF and graph-based sentence ranking. In 2025 13th International Conference on Intelligent Embedded, Micro-Electronics, Communication and Optical Networks (IEMECO… find
  7. Yimai Fang, Haoyue Zhu, Ewa Muszyńska, Alexander Kuhnle, and Simone Teufel. 2016. A proposition-based abstractive summariser. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 567–578, Osaka, Japan. find
  8. The COLING 2016 Organizing Committee. find
  9. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  10. Jessica Lin and Amir Zeldes. 2024. GUMsley: Evaluating entity salience in summarization for 12 English genres. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2575–2588, St. Julian’s, Malta. Association for…
  11. Jessica Lin and Amir Zeldes. 2025. GUM-SAGE: A novel dataset and approach for graded entity salience prediction. In Findings of the Association for Computational Linguistics: ACL 2025, pages 438–455, Vienna, Austria. Association for Computational Linguistics.
  12. Jessica Lin and Amir Zeldes. 2026. Expect the unexpected? Testing the surprisal of salient entities. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), San Diego, CA. Association for Computational Linguistics. find
  13. Yang Janet Liu and Amir Zeldes. 2023. GUMSum: Multi-genre data and evaluation for English abstractive summarization. In Findings of the Association for Computational Linguistics: ACL 2023, pages 9315–9327, Toronto, Canada. Association for Computational Linguistics.
  14. Zhengzhong Liu, Chenyan Xiong, Teruko Mitamura, and Eduard Hovy. 2018. Automatic event salience identification. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1226–1236, Brussels, Belgium. Association for Computational Linguistics. find
  15. William C. Mann and Sandra A. Thompson. 1988. Rhetorical Structure Theory: Toward a functional theory of text organization. Text, 8(3):243–281.
  16. Ramesh Nallapati, Feifei Zhai, and Bowen Zhou. 2017. SummaRuNNer: A recurrent neural network based sequence model for extractive summarization of documents. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17), pages 3075–3081, San Francisco, CA.
  17. Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018. Ranking sentences for extractive summarization with reinforcement learning. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Paper…
  18. Ani Nenkova and Rebecca Passonneau. 2004. Evaluating content selection in summarization: The pyramid method. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004, pages 145–152, Boston, Massachusetts… find
  19. Minh Phuong Nguyen and The Anh Le. 2024. Feature-based unsupervised method for salient sentence ranking in text summarization task. In Proceedings of the 2024 9th International Conference on Intelligent Information Technology, ICIIT ’24, page 346–351, New York, NY, USA. Association for Computing Mac… find
  20. Mihir Parmar, Hanieh Deilamsalehy, Franck Dernoncourt, Seunghyun Yoon, Ryan A. Rossi, and Trung Bui. 2024. Towards enhancing coherence in extractive summarization: Dataset and experiments with LLMs. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 1981… find
  21. Horacio Saggion and Thierry Poibeau. 2013. Automatic text summarization: Past, present and future. In Thierry Poibeau, Horacio Saggion, Jakub Piskorski, and Roman Yangarber, editors, Multi-source, Multilingual Information Extraction and Summarization, pages 3–21. Springer Berlin Heidelberg, Berlin,… find
  22. Tom Trabasso and Linda L. Sperry. 1985. Causal relatedness and importance of story events. Journal of Memory and Language, 24(5):595–611. find
  23. Salvatore Trani, Claudio Lucchese, R. Perego, David E. Losada, Diego Ceccarelli, and Salvatore Orlando. 2018. SEL: A unified algorithm for salient entity linking. Computational Intelligence, 34:2–29. find
  24. Florian Wolf and Edward Gibson. 2004. Paragraph-, word- and coherence-based approaches to sentence ranking: A comparison of algorithm and human performance. In Text Summarization Branches Out, page 18, Barcelona, Spain. Association for Computational Linguistics. find
  25. Amir Zeldes. 2017. The GUM corpus: Creating multilayer resources in the classroom. Language Resources and Evaluation, 51(3):581–612. find
  26. Amir Zeldes, Tatsuya Aoyama, Yang Janet Liu, Siyao Peng, Debopam Das, and Luke Gessler. 2025. eRST: A signaled graph theory of discourse relations and organization. Computational Linguistics, 51(1):23– 72.
  27. Amir Zeldes and Jessica Lin. 2026. What makes an entity salient in discourse? Corpus Linguistics and Linguistic Theory. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zeldes-2026-not-worth-mentioning-pilot-study/