Elamur Research⌕
☰
home / papers / 2017 / Automatic Negation and Speculation Detection in Veterinary Clinical Text

Automatic Negation and Speculation Detection in Veterinary Clinical Text

Paper · 2017 · Katherine Cheng, Timothy Baldwin and Karin Verspoor

Summary

The paper tackles negation and speculation detection in veterinary clinical notes from the VetCompass project. Clinicians search these records with Boolean keyword queries, and the results are cluttered with mentions that are negated ("no skin tent") or only hypothesised ("poss bacterial infection"). The authors carry the BioScope annotation scheme, which pairs cues with contiguous scopes that contain the cue, over to veterinary general-practice notes. These notes are written quickly and mostly for the author's own use, so they are terse, full of abbreviations, poorly punctuated and often misspelled. The authors annotate a new corpus drawn from VetCompass UK records. They then reimplement a simple CRF pipeline that first finds cues and then labels scope, and they train it on BioScope clinical data, VetCompass data, or both, sometimes oversampling the in-domain part. Out-of-domain training helps somewhat, mainly for negation. Reasonable performance, especially for speculation, still requires in-domain annotations. Recall stays the main weakness, which matters because the target use is filtering search results.

Key points

  • The corpus is built from a random sample of 1 million VetCompass UK clinical records. Notes range from free-text observations to billing-only entries.
  • Some cues appear only in this domain:
    • "nad" (no abnormalities detected / no acute distress);
    • question marks as speculation cues ("gastritis??");
    • shortenings such as "poss" and "susp";
    • informal hedges such as "feels like" and "looks like";
    • misspelled cues such as "wuithout" and "doestn".
  • Speculation cue detection (P/R/F):
    • LingScope baseline: 43.3/27.6/33.7;
    • CRF trained on BioScope only: 19.7/24.8/21.9;
    • CRF trained on VetCompass only: 88.7/44.8/59.5;
    • BioScope + VetCompass×5: 81.4/54.3/65.1 (best).
  • Speculation scope detection:
    • best result is CRF (POS; replace cue) on BioScope + VetCompass×5, at P 75.0, R 43.3, F 54.9;
    • the best LingScope variant reaches F 33.3.
  • Adding BioScope to VetCompass training raises recall but lowers precision for speculation. For both negation and speculation, recall stays well below precision.

Interesting ideas

  • I1. When pooling a large out-of-domain corpus with a small in-domain one, duplicating the in-domain portion (×2, ×5) steadily improved F-scores, which makes this a cheap weighting trick for domain transfer.
  • I2. The authors suggest catching unseen or misspelled cue variants ("doestn", "susp") through string similarity to a known cue list, rather than relying on exact word forms.
  • I3. Scope is defined as a contiguous span that must include its cue, and error analysis shows models stopping early on long scopes or on scopes anchored at "?". This makes boundary errors a distinct, measurable failure type.
  • I4. The paper's motivation is suppressing negated or speculative mentions before they count as real occurrences. In other words, polarity and certainty gate what gets asserted.
  • I5. Ambiguous items such as "not sure" (a speculation cue that contains "not") and "not bad" (not a real negation) caused confusion between the two detectors. Multiword cues therefore need to be resolved jointly rather than word by word.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): When silver data from a general treebank is mixed with a small set of agreed in-domain sentences (tales, school tests), the in-domain part could be oversampled, and the ×1/×2/×5 curve tracked as a standard ablation.
  • I2 (tokens): The tagger/lemmatizer could flag near-matches to a closed negation/hedge lexicon, so noisy forms still receive a polarity feature that the explanation trace can display.
  • I3 (frames): Negation/modality scope can be checked against the UD subtree of the cue's head, and both scope boundaries reported separately, so truncated scopes show up in diagnostics.
  • I4 (world): Entity states and events taken from "no X" or "possibly X" clauses should be stored as negated or hypothetical facts rather than plain ones, so the reader does not answer from a hedge.
  • I5 (frames): A small multiword cue table, consulted before single-word negation rules, could map "not sure" to uncertainty and "not bad" to positive evaluation, which also helps the implicature module.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 9,085 worksArtificial Intelligence · 8,888 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksBiochemistry, Genetics and Molecular Biology · 116 worksMolecular Biology · 114 works

Cite

Cite as Katherine Cheng, Timothy Baldwin and Karin Verspoor. 2017. Automatic Negation and Speculation Detection in Veterinary Clinical Text. Proceedings of the Australasian Language Technology Association Workshop 2017.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{cheng2017automatic,
  title = {{Automatic Negation and Speculation Detection in Veterinary Clinical Text}},
  author = {Katherine Cheng, Timothy Baldwin and Karin Verspoor},
  booktitle = {Proceedings of the Australasian Language Technology Association Workshop 2017},
  year = {2017},
  url = {https://aclanthology.org/U17-1008/}
}

References

21 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Shashank Agarwal and Hong Yu. 2010a. Biomedical negation scope detection with conditional random fields. Journal of the American Medical Informatics Association 17(6):696–701. ↗
  2. Shashank Agarwal and Hong Yu. 2010b. Detecting hedge cues and their scope in biomedical text with conditional random fields. Journal of Biomedical Informatics 43(6):953–961. ↗
  3. Timothy Miller, Steven Bethard, Hadi Amiri, and Guergana Savova. 2017. Unsupervised domain adaptation for clinical negation detection. BioNLP 2017 pages 165–170. find
  4. R Michele Anholt, John Berezowski, Iqbal Jamal, Carl Ribble, and Craig Stephen. 2014. Mining free-text medical records for companion animal enteric syndrome surveillance. Preventive Veterinary Medicine 113(4):417–422. ↗
  5. Roser Morante and Walter Daelemans. 2009a. Learning the scope of hedge cues in biomedical texts. In Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing. Association for Computational Linguistics, pages 28–36.
  6. Wendy W Chapman, Will Bridewell, Paul Hanbury, Gregory F Cooper, and Bruce G Buchanan. 2001. A simple algorithm for identifying negated findings and diseases in discharge summaries. Journal of biomedical informatics 34(5):301–310. ↗
  7. Roser Morante and Walter Daelemans. 2009b. A metalearning approach to processing the scope of negation. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning. Association for Computational Linguistics, pages 21– 29. find
  8. Nigel Collier, Hyun Seok Park, Norihiro Ogata, Yuka Tateishi, Chikashi Nobata, Tomoko Ohta, Tateshi Sekimizu, Hisao Imai, Katsutoshi Ibushi, and Junichi Tsujii. 1999. The genia project: corpus-based knowledge acquisition and information extraction from genome research papers. In Proceedings of the ninth conference on European chapter of the Association for Computational Linguistics. Association for Computational Linguistics, pages 271–272. find
  9. Roser Morante, Anthony Liekens, and Walter Daelemans. 2008. Learning the scope of negation in biomedical texts. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, pages 715–724. find
  10. Pradeep G Mutalik, Aniruddha Deshpande, and Prakash M Nadkarni. 2001. Use of general-purpose negation detection to augment concept indexing of medical documents: a quantitative study using the umls. Journal of the American Medical Informatics Association 8(6):598–609. ↗
  11. Noa P Cruz Dı́az, Manuel J Maña López, Jacinto Mata Vázquez, and Victoria Pachón Álvarez. 2012. A machine-learning approach to negation and speculation detection in clinical texts. Journal of the Association for Information Science and Technology 63(7):1398–1410. find
  12. Jonathon Read, Rebecca Dridan, Stephan Oepen, and Lars Jørgen Solberg. 2012. Sentence boundary detection: A long solved problem? COLING (Posters) 12:985–994.
  13. Haibo Ding and Ellen Riloff. 2015. Extracting information about medication use from veterinary discussions. In Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL. pages 1452–1458. find
  14. Burr Settles. 2005. Abner: an open source tool for automatically tagging genes, proteins and other entity names in text. Bioinformatics 21(14):3191–3192. ↗
  15. Marco Duz, John F Marshall, and Tim Parkin. 2017. Validation of an improved computer-assisted technique for mining free-text electronic medical records. JMIR Medical Informatics 5(2):e17. ↗
  16. Pontus Stenetorp, Sampo Pyysalo, Goran Topić, Tomoko Ohta, Sophia Ananiadou, and Jun’ichi Tsujii. 2012. Brat: a web-based tool for nlp-assisted text annotation. In Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, pages 102–107.
  17. John Lafferty, Andrew McCallum, and Fernando CN Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning. pages 282– 289. find
  18. György Szarvas, Veronika Vincze, Richárd Farkas, and János Csirik. 2008. The bioscope corpus: annotation for negation, uncertainty and their scope in biomedical texts. In Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing. Association for Computational Linguistics, pages 38–45.
  19. K Lam, Tim Parkin, Christopher Riggs, and Kenton Morgan. 2007. Use of free text clinical records in identifying syndromes and analysing health data. Veterinary Record 161(16):547–51. ↗
  20. Paul McGreevy, Peter Thomson, Navneet Dhand, David Raubenheimer, Sophie Masters, Caroline Mansfield, Tim Baldwin, Ricardo Soares Magalhaes, Jacquie Rand, Peter Hill, Anne Peaston, James Gilkerson, Martin Combs, Shane Raidal, Peter Irwin, Peter Irons, Richard Squires, David Brodbelt, and Jeremy Hammond. 2017. VetCompass Australia: Big data and real-time surveillance for veterinary science. Animals 7(10). find
  21. Stephen Wu, Timothy Miller, James Masanz, Matt Coarr, Scott Halgrim, David Carrell, and Cheryl Clark. 2014. Negation’s not solved: generalizability versus optimizability in clinical natural language processing. PloS one 9(11):e112774. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/cheng-2017-automatic-negation-speculation-detection-veterinary/