📄 Learning to Detect Hedges and their Scope Using CRF

Summary

The paper describes a CoNLL-2010 shared task entry that identifies speculative language (hedges) in biomedical writing. It covers both subtasks: flagging uncertain sentences and finding the span each hedge cue governs. The authors treat cue detection as sequence labeling with a CRF (CRF++ 0.51). Features are word, POS, lemma and chunk tags from GENIA Tagger, plus a new HCDic feature. HCDic is a hedge cue dictionary built from the BioScope abstracts, and each entry is graded by how often it occurs outside speculative sentences. A sentence counts as uncertain if any cue is found in it. A second CRF marks only the first and last token of each scope, and simple rules then guarantee exactly one scope per cue. On BioScope, HCDic lifts cue-level F-score from about 0.68 to 0.75, and the sentence-level task reaches roughly 84%. Scope detection stays weak (F about 37.6%), and the method transfers poorly to Wikipedia weasel words. The work shows both how useful and how limited a small, graded lexicon can be for a closed cue vocabulary.

Key points

• BioScope abstracts contain 143 distinct hedge cues, averaging 1.44 words each and at most 4 words. A typical speculative sentence has about one cue (1.28 per sentence).
• HCDic grades cues into four types (L, H, FL, FH) using the non-speculative/speculative occurrence ratio. FL and FH are split at a threshold of 1.0, and cues seen more than 5 times only in speculative sentences become H.
• At cue level on BioScope, all features give P 0.7671 / R 0.7393 / F 0.7529, compared with F 0.6781 without HCDic. Task 1 sentence-level F is about 0.84.
• On Wikipedia, recall falls to 0.2001 (F 0.3120), even after the weasel dictionary is compressed from 1984 cues to 170 adjective "core words".
• Task 2 scope results are P 34.8%, R 41%, F 37.6%. About 86.1% of scopes start at the cue itself, and more than 80% end at the sentence end.

Interesting ideas

• I1. Instead of a binary dictionary hit, give each lexicon entry a reliability grade based on how often the word appears in non-target contexts, and feed that grade to the learner as a feature. This lets the model concentrate on ambiguous words like "or" versus near-certain ones like "suggesting".
• I2. Cast scope detection as predicting only the boundary tokens (start and end) rather than labeling every token inside the span. Then repair the output with deterministic rules: keep the nearer head and the farther tail, and fall back to "cue to sentence end".
• I3. When lexical cues are mostly multiword and rare, as with weasel phrases like "many persons", collapse them to a recurring core word to keep dictionary coverage.
• I4. Cue-level errors largely disappear at sentence level: one found cue is enough, and a false cue in an already speculative sentence still gives the right answer. Evaluating at several granularities shows where errors actually matter.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): A graded hedge/modal lexicon could mark frame assertions as asserted, likely or uncertain, with the grade recorded as the explanation.
• I2 (tree): Hedge scope could be projected from the UD subtree of the cue verb or modal, with these fallback rules as an explainable backup when the parse is unsure.
• I3 (world): Vague quantifiers and attribution phrases reduced to core words could tag world-model facts as having unattributed or vague sources.
• I4 (learn): Report hedge or modality accuracy at both the token and the proposition level, so downstream answer quality is not misjudged from token scores alone.