Elamur Research⌕
☰
home / papers / 2010 / Identifying the Information Structure of Scientific Abstracts: An Investigation of Three Different Schemes

Identifying the Information Structure of Scientific Abstracts: An Investigation of Three Different Schemes

Summary

Many text mining and reading tasks need particular kinds of content from scientific papers, such as aims, methods or findings. Most schemes for labelling this information structure were built for full articles, so it was unclear whether they suit abstracts. The authors compared three schemes of increasing granularity on biomedical abstracts used in cancer risk assessment (CRA): section names (S1, 4 categories), a reduced Argumentative Zoning scheme (S2, 7 categories), and the first layer of Core Scientific Concepts (CoreSC; S3, 11 categories). They annotated 1000 MedLine abstracts with each scheme at the sentence level, measured inter-annotator agreement, and analysed how the schemes overlap. They then trained Naive Bayes and SVM classifiers to assign the categories automatically. Even for the most detailed scheme, most categories occur in abstracts and SVM recognises them fairly reliably. This suggests that fine-grained schemes are practical for abstracts. The resulting annotated corpus also allows later task-based evaluation within CRA.

Key points

  • The data are 1000 abstracts randomly selected from the CRA corpus of Korhonen et al. (2009), totalling 7,985 sentences and 225,785 words. A single annotator labelled the corpus with each scheme, taking 45, 50 and 90 hours for S1, S2 and S3.
  • On a 300-abstract subset with three annotators, Cohen's Kappa was 0.84 for S1, 0.85 for S2 and 0.50 for S3. S3 agreement was highest between the domain expert and the linguist (κ = 0.60).
  • The schemes partly subsume one another. S3 categories predict S1 and S2 labels well, with lambda values of 0.86 and 0.84.
  • SVM gave the best results:
    • S1: 89% accuracy across all 4 categories.
    • S2: 90% accuracy, with 6 of 7 categories found; REL was missed.
    • S3: 81% accuracy, with 9 of 11 categories found; HYP and MOD were missed.
  • The best feature set used all features except Verb. Planned next steps include larger training data, better use of sentence sequence, comparison with CRF and Maximum Entropy, and evaluation on CRA tasks.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{guo2010identifying,
  title = {{Identifying the Information Structure of Scientific Abstracts: An Investigation of Three Different Schemes}},
  author = {Guo, Yufan and Korhonen, Anna and Liakata, Maria and Silins, Ilona and Sun, Lin and Stenius, Ulla},
  booktitle = {Proceedings of the 2010 Workshop on Biomedical Natural Language Processing},
  year = {2010},
  url = {https://aclanthology.org/W10-1913/}
}

References

22 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. J. Cohen. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20:37–46. find
  2. J. R. Curran, S. Clark, and J. Bos. 2007. Linguistically motivated large-scale nlp with c&c and boxer. In Proceedings of the ACL 2007 Demonstrations Session, pages 33–36. find
  3. K. Hirohata, N. Okazaki, S. Ananiadou, and M. Ishizuka. 2008. Identifying sections in scientific abstracts using conditional random fields. In Proc. of 3rd International Joint Conference on Natural Language Processing. find
  4. A. Korhonen, L. Sun, I. Silins, and U. Stenius. 2009. The first step in the development of text mining technology for cancer risk assessment: Identifying and organizing scientific evidence in risk assessment literature. BMC Bioinformatics, 10:303. find
  5. J. Lafferty, A. McCallum, and F. Pereira. 2001. Con-ditionl random fields: Probabilistic models for segmenting and labeling sequence data. In Proc. of ICML. find
  6. J. R. Landis and G. G. Koch. 1977. The measurement of observer agreement for categorical data. Biometrics, 33:159–174. find
  7. M. Liakata and L.N. Soldatova. 2008. Guidelines for the annotation of general scientific concepts. Aberystwyth University, JISC Project Report http://ie-repository.jisc.ac.uk/88/. ↗
  8. M. Liakata, Claire Q, and L.N. Soldatova. 2009. Semantic annotation of papers: Interface & enrichment tool (sapient). In Proceedings of BioNLP-09, pages 193–200, Boulder, Colorado. find
  9. M. Liakata, S. Teufel, A. Siddharthan, and C. Batchelor. 2010. Corpora for the conceptualisation and zoning of scientific papers. To appear in the 7th International Conference on Language Resources and Evaluation. find
  10. J. Lin, D. Karakos, D. Demner-Fushman, and S. Khudanpur. 2006. Generative content models for structural analysis of medical abstracts. In Proceedings of BioNLP-06, pages 65–72, New York, USA. find
  11. J. Lin. 2009. Is searching full text more effective than searching abstracts? BMC Bioinformatics, 10:46. find
  12. S. Merity, T. Murphy, and J. R. Curran. 2009. Accurate argumentative zoning with maximum entropy models. In Proceedings of the 2009 Workshop on Text and Citation Analysis for Scholarly Digital Libraries, pages 19–26. Association for Computational Linguistics. find
  13. Y. Mizuta, A. Korhonen, T. Mullen, and N. Collier. 2005. Zone analysis in biology articles as a basis for information extraction. International Journal of Medical Informatics on Natural Language Processing in Biomedicine and Its Applications. find
  14. T. Mullen, Y. Mizuta, and N. Collier. 2005. A baseline feature set for learning rhetorical zones using full articles in the biomedical domain. Natural language processing and text mining, 7:52–58. find
  15. P. Ruch, C. Boyer, C. Chichester, I. Tbahriti, A. Geiss-buhler, P. Fabry, J. Gobeill, V. Pillet, D. Rebholz-Schuhmann, C. Lovis, and A. L. Veuthey. 2007. Using argumentation to extract key sentences from biomedical abstracts. Int J Med Inform, 76:195– 200. find
  16. H. Shatkay, F. Pan, A. Rzhetsky, and W. J. Wilbur. 2008. Multi-dimensional classification of biomedical text: Toward automated, practical provision of high-utility text to diverse users. Bioinformatics, 18:2086–2093. find
  17. S. Siegel and N. J. Jr. Castellan. 1988. Nonparametric Statistics for the Behavioral Sciences. McGraw-Hill, Berkeley, CA, 2nd edition. find
  18. L. Sun and A. Korhonen. 2009. Improving verb clustering with automatically acquired selectional preference. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. find
  19. I. Tbahriti, C. Chichester, Frederique Lisacek, and P. Ruch. 2006. Using argumentation to retrieve articles with similar citations. Int J Med Inform, 75:488–495. find
  20. S. Teufel and M. Moens. 2002. Summarizing scientific articles: Experiments with relevance and rhetorical status. Computational Linguistics, 28:409–445. find our card
  21. S. Teufel, A. Siddharthan, and C. Batchelor. 2009. Towards domain-independent argumentative zoning: Evidence from chemistry and computational linguistics. In Proc. of EMNLP. find
  22. I. H. Witten, 2008. Data mining: practical machine learning tools and techniques with Java Implementations. http://www.cs.waikato.ac.nz/ml/weka/. ↗

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/guo-2010-identifying-information-structure-scientific-abstracts/