📄 Using Hedges to Enhance a Disease Outbreak Report Text Mining System

Summary

Systems that watch online news for early signs of infectious disease outbreaks must separate real outbreak reports from articles that only mention a disease, such as stories about vaccination campaigns or research. The authors observe that outbreak news tends to be tentative, because early reports describe situations that are not yet confirmed. They apply this observation to the BioCaster system in two ways. First, they add a single binary hedge feature to BioCaster's n-gram topic classifier. The feature fires when a word from the Mercer et al. (2004) hedge lexicon appears within five words of a disease named entity. They test it on the BioCaster corpus of 1000 annotated news articles. For an SVM, the feature raises accuracy modestly, though a larger χ² 9000 n-gram feature set still performs best. Second, they build a "speculative" score. A document's hedge density is placed against tertile cut-points calculated from 10,000 Reuters documents, which yields a LOW, MEDIUM or HIGH tag. Relevant outbreak reports fall into HIGH much more often than rejected articles do. This suggests that uncertainty markers carry genuine signal for surveillance.

Key points

• The BioCaster corpus has 1000 news articles from sources such as BBC, the New York Times and ProMED-Mail. Each article is labelled relevant (350) or reject (650) and annotated with named entities. Borderline cases were included on purpose, and the corpus is not publicly released.

• The hedge lexicon is adapted from Mercer et al. (2004). It is described as 72 verbs (including inflected forms) plus 32 non-verbs, and elsewhere in the paper as 105 lexemes. A χ² test showed that hedge frequency differs between the two classes at P < 0.01. The most discriminating hedges include reported, suspected, probable and suspect.

• Classification results:

  • With an SVM, unigrams give 90.9% accuracy (F 0.87), and unigrams plus the hedge feature give 91.7% (F 0.89).
  • With Naive Bayes, adding the hedge feature does not help (88.4 → 88.0).
  • The χ² 9000 n-gram set with Naive Bayes is the best result at 94.8% (F 0.93).

• The speculative metric uses length-normalised hedge percentages from 10,000 Reuters documents to set tertile boundaries. LOW is ≤0.2544%, MEDIUM is up to about 1.0574%, and HIGH is anything above that.

• Distribution of the speculative tag by class:

  Tag	Relevant articles	Rejected articles
  HIGH	64.2%	48.3%
  MEDIUM	29.5%	36.7%
  LOW	6.3%	15.0%

Interesting ideas

• I1. Anchor hedges to an entity. A hedge only counts when it occurs near the entity that matters (a disease mention), so general vagueness elsewhere in the text is ignored and precision improves.
• I2. Calibrate a scale on a reference corpus. Graded labels come from tertiles of a feature's distribution in a general reference corpus, not from thresholds picked by hand. Each label then has a clear, reproducible meaning.
• I3. Treat speculative news as a genre signal. Early, unconfirmed reports about events are marked by hedging, so the density of hedges says something about what kind of text a document is.
• I4. Test one interpretable feature in isolation. Adding a single interpretable feature to a baseline, and checking it across two learners, shows whether that linguistic cue adds real value.
• I5. Rank words with χ² within a fixed lexicon. Running χ² selection inside a known hedge lexicon shows which entries actually discriminate in a domain; here, reported and suspected dominate in news.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): Uncertainty markers could be attached to the specific frame whose argument they are near, instead of the whole sentence. This would give a local, explainable epistemic flag on "who did what to whom."
• I2 (world): Certainty or speculativeness levels attached to world states could be calibrated against a neutral news corpus, so that "highly speculative" is a documented percentile rather than an arbitrary number.
• I3 (answer): A reader answering questions could use hedge density to soften its answers ("reportedly", "suspected") when the source text is tentative, rather than stating claims as established facts.
• I4 (learn): Each new hand-built cue could be evaluated as one ablation against a baseline under more than one model, as a control against attributing gains to the cue by mistake.
• I5 (tokens): Lemma-level markers in the en crate could carry a domain-weighted strength, with the weights derived from similar χ² rankings on target corpora.