Elamur Research
☰
home / papers / 2002 / Summarizing Scientific Articles: Experiments with Relevance and Rhetorical Status

Summarizing Scientific Articles: Experiments with Relevance and Rhetorical Status

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" does not allow redistributing the text itself. Original — aclanthology.org/J02-4002.pdf

Summary

Standard sentence extraction works poorly for scientific papers. Their key contribution is often buried late in the text, and summaries must shrink a long article to a tiny fraction of its length, so extracted sentences lose their discourse context. A single sentence could state the paper's own solution or describe a rival approach the authors reject. Teufel and Moens address this by labelling each extracted sentence with its rhetorical role. They build a gold standard from computational linguistics conference papers, with human annotations of each sentence's relevance and rhetorical status, and measure how consistently judges agree. A system trained on this data then picks content from new articles and assigns it to one of seven rhetorical categories. The resulting labelled extracts already serve as summaries, and an extrinsic study found them useful as document surrogates. They can also feed later generation of summaries tailored to a user's expertise or task, such as background-heavy overviews for newcomers or contrast-focused ones for experts.

Key points

  • Corpus: conference papers from COLING, ANLP and (E)ACL events. The chronologically first 80 articles were used to design the annotation scheme.
  • Seven rhetorical categories: examples include AIM, BASIS and CONTRAST. The categories cover three aspects:
    • problem solving (goal and contribution)
    • intellectual attribution, traced through the sequence of grammatical subjects ("agent structure")
    • links to other work, signalled by indicator phrases, section headers and citations
  • Compression: condensing a 20-page article to half a page means keeping about 2.5% of the text. For news stories, 20–30% is typical.
  • Running example: Pereira, Tishby and Lee, "Distributional Clustering of English Words" (ACL-1993, cmp-lg/9408011), the most-cited paper in the collection. It is used to show a non-expert summary and an expert, contrastive summary.
  • Missing results: the supplied excerpt leaves out the middle of the paper. Inter-annotator agreement figures and classifier accuracy are therefore not given here.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing

Cited by

137 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 134 not yet.

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/089120102762671936

@article{teufel2002summarizing,
  title = {{Summarizing Scientific Articles: Experiments with Relevance and Rhetorical Status}},
  author = {Teufel, Simone and Moens, Marc},
  journal = {Computational Linguistics},
  year = {2002},
  doi = {10.1162/089120102762671936},
  url = {https://doi.org/10.1162/089120102762671936}
}

References

69 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Barzilay, Regina, Michael Collins, Julia Hirschberg, and Steve Whittaker. 2000. The rules behind roles. In Proceedings of AAAI-00. find
  2. Barzilay, Regina, Kathleen R. McKeown, and Michael Elhadad. 1999. Information fusion in the context of multi-document summarization. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics (ACL-99), pages 550–557. find
  3. Baxendale, Phyllis B. 1958. Man-made index for technical literature—An experiment. IBM Journal of Research and Development, 2(4):354–361. find
  4. Biber, Douglas. 1995. Dimensions of Register Variation: A Cross-Linguistic Comparison. Cambridge University Press, Cambridge, England. find
  5. Brandow, Ronald, Karl Mitze, and Lisa F. Rau. 1995. Automatic condensation of electronic publications by sentence selection. Information Processing and Management, 31(5):675–685. find
  6. Carletta, Jean. 1996. Assessing agreement on classification tasks. The kappa statistic. Computational Linguistics, 22(2):249–254. find our card
  7. Choi, Freddy Y. Y. 2000. Advances in domain independent linear text segmentation. In Proceedings of the Sixth Applied Natural Language Conference (ANLP-00) and the First Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL-00), pages 26–33. find
  8. CMP LG. 1994. The computation and language e-print archive. http://xxx.lanl.gov/cmp-lg. ↗
  9. Dunning, Ted. 1993. Accurate methods for the statistics of surprise and coincidence. Computational Linguistics, 19(1):61–74. find our card
  10. Edmundson, H. P. 1969. New methods in automatic extracting. Journal of the Association for Computing Machinery, 16(2):264–285. find
  11. Edmundson, H. P., and Wyllys, R. E. 1961. Automatic abstracting and indexing—Survey and recommendations. Communications of the ACM, 4(5):226–234. find
  12. Garfield, Eugene. 1979. Citation Indexing: Its Theory and Application in Science, Technology and Humanities. J. Wiley, New York. find
  13. Grefenstette, Gregory. 1998. Producing intelligent telegraphic text reduction to provide an audio scanning service for the blind. In D. R. Radev and E. H. Hovy, editors, Working Notes of the AAAI Spring Symposium on Intelligent Text Summarization, pages 111–117. find
  14. Grover, Claire, Andrei Mikheev, and Colin Matheson. 1999. LT TTT version 1.0: Text tokenisation software. Technical Report, Human Communication Research Centre, University of Edinburgh. http://www.ltg.ed.ac.uk/software/ttt/. ↗
  15. Hearst, Marti A. 1997. Texttiling: Segmenting text into multi-paragraph subtopic passages. Computational Linguistics, 23(1):33–64. find
  16. Hyland, Ken. 1998. Persuasion and context: The pragmatics of academic metadiscourse. Journal of Pragmatics, 30(4):437–455. find
  17. Jing, Hongyan, Regina Barzilay, Kathleen R. McKeown, and Michael Elhadad. 1998. Summarization evaluation methods: Experiments and analysis. In D. R. Radev and E. H. Hovy, editors, Working Notes of the AAAI Spring Symposium on Intelligent Text Summarization, pages 60–68. find
  18. Jing, Hongyan and Kathleen R. McKeown. 2000. Cut and paste based summarization. In Proceedings of the Sixth Applied Natural Language Conference (ANLP-00) and the First Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL-00), pages 178–185. find
  19. Jordan, Michael P. 1984. Rhetoric of Everyday English Texts. Allen and Unwin, London. find
  20. Klavans, Judith L. and Min-Yen Kan. 1998. Role of verbs in document analysis. In Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and the 17th International Conference on Computational Linguistics (ACL/COLING-98), pages 680–686. find
  21. Knight, Kevin and Daniel Marcu. 2000. Statistics-based summarization—Step one: Sentence compression. In Proceedings of the 17th National Conference of the American Association for Artificial Intelligence (AAAI-2000), pages 703–710. find
  22. Krippendorff, Klaus. 1980. Content Analysis: An Introduction to Its Methodology. Sage Publications, Beverly Hills, CA. find
  23. Kupiec, Julian, Jan O. Pedersen, and Francine Chen. 1995. A trainable document summarizer. In Proceedings of the 18th Annual International Conference on Research and Development in Information Retrieval (SIGIR-95), pages 68–73. find
  24. Lancaster, Frederick Wilfrid. 1998. Indexing and Abstracting in Theory and Practice. Library Association, London. find
  25. Landis, J. Richard and Gary G. Koch. 1977. The measurement of observer agreement for categorical data. Biometrics, 33:159–174. find
  26. Lawrence, Steve, C. Lee Giles, and Kurt Bollacker. 1999. Digital libraries and autonomous citation indexing. IEEE Computer, 32(6):67–71. find
  27. Levin, Beth. 1993. English Verb Classes and Alternations. University of Chicago Press, Chicago. find
  28. Lewis, David D. 1991. Evaluating text categorisation. In Speech and Natural Language: Proceedings of the ARPA Workshop of Human Language Technology. find
  29. Liddy, Elizabeth DuRoss. 1991. The discourse-level structure of empirical abstracts: An exploratory study. Information Processing and Management, 27(1):55–81. find
  30. Lin, Chin-Yew and Eduard H. Hovy. 1997. Identifying topics by position. In Proceedings of the Fifth Applied Natural Language Conference (ANLP-97), pages 283–290. find
  31. Luhn, Hans Peter. 1958. The automatic creation of literature abstracts. IBM Journal of Research and Development, 2(2):159–165. find
  32. Mani, Inderjeet, Therese Firmin, David House, Gary Klein, Beth Sundheim, and Lynette Hirschman. 1999. The TIPSTER SUMMAC text summarization evaluation. In Proceedings of the Ninth Meeting of the European Chapter of the Association for Computational Linguistics (EACL-99), pages 77–85. find
  33. Mani, Inderjeet, Barbara Gates, and Eric Bloedorn. 1999. Improving summaries by revising them. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics (ACL-99), pages 558–565. find
  34. Mann, William C. and Sandra A. Thompson. 1987. Rhetorical structure theory: Description and construction of text structures. In Gerard Kempen, editor, Natural Language Generation: New Results in Artificial Intelligence, Psychology, and Linguistics. Martinus Nijhoff Publishers, Dordrecht, the Netherl… find
  35. Marcu, Daniel. 1999. Discourse trees are good indicators of importance in text. In I. Mani and M. T. Maybury, editors, Advances in Automatic Text Summarization. MIT Press, Cambridge, pages 123–136. find
  36. McCallum, Andrew. 1997. Training algorithms for linear text classifiers. In Proceedings of the 19th Annual International Conference on Research and Development in Information Retrieval (SIGIR-97). find
  37. Mizzaro, Stefano. 1997. Relevance: The whole history. Journal of the American Society for Information Science, 48(9):810–832. find
  38. Morris, Jane and Graeme Hirst. 1991. Lexical cohesion computed by thesaural relations as an indicator of the structure of text. Computational Linguistics, 17(1):21–48. find our card
  39. Myers, Greg. 1992. In this paper we report . . . —Speech acts and scientific facts. Journal of Pragmatics, 17(4):295–313. find
  40. Nanba, Hidetsugu and Manabu Okumura. 1999. Towards multi-paper summarization using reference information. In Proceedings of IJCAI-99, pages 926–931. find
  41. Paice, Chris D. 1990. Constructing literature abstracts by computer: Techniques and prospects. Information Processing and Management, 26(1):171–186. find
  42. Paice, Chris D. and A. Paul Jones. 1993. The identification of important concepts in highly structured technical papers. In Proceedings of the 16th Annual International Conference on Research and Development in Information Retrieval (SIGIR-93), pages 69–78. find
  43. Pollock, Joseph J. and Antonio Zamora. 1975. Automatic abstracting research at the chemical abstracts service. Journal of Chemical Information and Computer Sciences, 15(4):226–232. find
  44. Radev, Dragomir R. and Kathleen R. McKeown. 1998. Generating natural language summaries from multiple on-line sources. Computational Linguistics, 24(3):469–500. find
  45. Rath, G. J., A. Resnick, and T. R. Savage. 1961. The formation of abstracts by the selection of sentences. American Documentation, 12(2):139–143. find
  46. Resnik, Philip and Mona Diab. 2000. Measuring verb similarity. In Twenty-Second Annual Meeting of the Cognitive Science Society (COGSCI2000). find
  47. Riley, Kathryn. 1991. Passive voice and rhetorical role in scientific writing. Journal of Technical Writing and Communication, 21(3):239–257. find
  48. Rowley, Jennifer. 1982. Abstracting and Indexing. Bingley, London. find
  49. Saggion, Horacio and Guy Lapalme. 2000. Selective analysis for automatic abstracting: Evaluating indicativeness and acceptability. In Proceedings of Content-Based Multimedia Information Access (RIAO), pages 747–764. find
  50. Salton, Gerard and Michael J. McGill. 1983. Introduction to Modern Information Retrieval. McGraw-Hill, Tokyo. find
  51. Samuel, Ken, Sandra Carberry, and K. Vijay-Shanker. 1999. Automatically selecting useful phrases for dialogue act tagging. In Proceedings of the Pacific Association for Computational Linguistics (PACLING-99). find
  52. Saracevic, Tefko. 1975. Relevance: A review of and a framework for the thinking on the notion in information science. Journal of the American Society for Information Science, 26(6):321–343. find
  53. Siegel, Sidney and N. John Castellan Jr. 1988. Nonparametric Statistics for the Behavioral Sciences. McGraw-Hill, Berkeley, CA, second edition. find
  54. Sparck Jones, Karen. 1990. What sort of thing is an AI experiment? In D. Partridge and Yorick Wilks, editors, The Foundations of Artificial Intelligence: A SourceBook. Cambridge University Press, Cambridge, pages 274–281. find
  55. Sparck Jones, Karen. 1999. Automatic summarising: Factors and directions. In I. Mani and M. T. Maybury, editors, Advances in Automatic Text Summarization. MIT Press, Cambridge, pages 1–12. find
  56. Swales, John, 1990. Research articles in English. In Genre Analysis: English in Academic and Research Settings. Cambridge University Press, Cambridge, chapter 7, pages 110–176. find
  57. Teufel, Simone, 1999. Argumentative Zoning: Information Extraction from Scientific Text. Ph.D. thesis, School of Cognitive Science, University of Edinburgh, Edinburgh. find
  58. Teufel, Simone. 2001. Task-based evaluation of summary quality: Describing relationships between scientific papers. In Proceedings of NAACL-01 Workshop “Automatic Text Summarization.” Teufel, Simone, Jean Carletta, and Marc Moens. 1999. An annotation scheme for discourse-level argumentation in resea… find
  59. Teufel, Simone and Marc Moens. 1997. Sentence extraction as a classification task. In Proceedings of the ACL/EACL-97 Workshop on Intelligent Scalable Text Summarization, pages 58–65. find
  60. Teufel, Simone and Marc Moens. 2000. What’s yours and what’s mine: Determining intellectual attribution in scientific text. In Proceedings of the Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora. find
  61. Thompson, Geoff and Ye Yiyun. 1991. Evaluation in the reporting verbs used in academic papers. Applied Linguistics, 12(4):365–382. find
  62. Tombros, Anastasios, and Mark Sanderson. 1998. Advantages of query biased summaries. In Proceedings of the 21st Annual International Conference on Research and Development in Information Retrieval (SIGIR-98). Association of Computing Machinery. find
  63. Trawinski, Bogdan. 1989. A methodology for writing problem-structured abstracts. Information Processing and Management, 25(6):693–702. van Dijk, Teun A. 1980. Macrostructures: An Interdisciplinary Study of Global Structures in Discourse, Interaction and Cognition. find
  64. Lawrence Erlbaum, Hillsdale, NJ. van Rijsbergen, Cornelis Joost. 1979. Information Retrieval. Butterworth, London, second edition. find
  65. Wiebe Janyce. 1994. Tracking point of view in narrative. Computational Linguistics, 20(2):223–287. find
  66. Yang, Yiming and Xin Liu. 1999. A re-examination of text categorization methods. In Proceedings of the 22nd Annual International Conference on Research and Development in Information Retrieval (SIGIR-99), pages 42–49. find
  67. Zappen, James P. 1983. A rhetoric for research in sciences and technologies. In Paul V. Anderson, R. John Brockman, and Carolyn R. Miller, editors, New Essays in Technical and Scientific Communication Research Theory Practice. Baywood, Farmingdale, NY, pages 123–138. find
  68. Zechner, Klaus. 1995. Automatic text abstracting by selecting relevant passages. Master’s thesis, Centre for Cognitive Science, University of Edinburgh, Edinburgh. find
  69. Ziman, John M. 1969. Information, communication, knowledge. Nature, 224:318–324. find