📄 Detecting Speculative Language Using Syntactic Dependencies and Logistic Regression

Summary

The paper presents the authors' entry to the CoNLL-2010 shared task on hedging, restricted to the biomedical track. There were two goals. The first was to flag sentences that contain uncertain statements (Task1). The second was to mark how far each hedge cue reaches in its sentence (Task2).

Training used BioScope: nine full articles and 1,273 abstracts. Testing used 15 full articles. Text was parsed with a biomedical-adapted C&C CCG parser, and the output was converted to Stanford Dependency collapsed format. Both tasks were framed as token-level decisions and solved with Bayesian logistic regression under a Laplace prior.

• Task1: each token is classified as cue or non-cue, and the sentence label follows from those decisions. The cue features are lexical, PoS, CCG supertag and dependency-context features. Syntax stands in for word meaning, which helps separate speculative from literal uses of the same word.
• Task2: each token is judged as inside or outside a given cue's scope, using the dependency path between them.

The system reached 85.21% F on Task1, which was competitive. It reached only 44.11% F on Task2. The authors analyse the errors and propose ways to make the annotation scheme cleaner.

Key points

• Data: BioScope corpus. Training had 9 full articles plus 1,273 abstracts. The test set was 15 full articles, so abstracts were used only for training.
• Dictionary baseline: the authors collected single-token cue lemmas from the training data, using morpha for lemmatisation. In four-fold cross-validation on the full articles, this list reached 98.07% recall but just 59.53% F. Coverage was good; precision was poor.
• Ambiguous cue words: "if" is a cue about 4% of the time, "appear" 83%, and "can" 16%. Even so, "can" ranks fifth by frequency among all cues found in the full articles.
• Pipeline: text was tokenised with the Penn TreeBank script, because the PoS tagger is trained on GENIA. Parsing used C&C with an in-domain retrained tagger and supertagger. Output was converted to SD, and the model was Laplace-prior logistic regression (Genkin et al.).
• What helped: choosing a suitable prior improved results. For scope, a hand-designed scheme for simplifying dependency paths gave a large gain by reducing sparsity.

Interesting ideas

• I1. Deciding whether a word acts as a hedge is treated like word sense disambiguation with just two senses shared by every word, and syntactic context is used to pick the sense.
• I2. Lexical features are paired with grammatical evidence, for example a verb's lemma with a clausal complement, a negation, a modifying adverb, or passive voice. Examples include "could be due" and "do not know". This matters because a linear model cannot learn such combinations by itself.
• I3. The CCG supertag holds subcategorisation information even when the full parse fails, so cue features survive parser breakdowns.
• I4. Scope is found by classifying every token against a cue through their dependency path, after the paths are simplified to fight sparsity. The authors suggest the simplification itself could be learned.
• I5. Diagnose first with a high-recall dictionary, then check whether the remaining problem is precision or coverage before adding features. Use a sparse Laplace prior when most features are expected to be useless.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): prag could label modal and epistemic tokens as hedging or literal from their UD context, so speech-act frames carry a certainty flag instead of a keyword match.
• I2 (frames): explicit rules over UD relations (ccomp, neg, advmod, nsubj:pass) could mark a frame as weakened or tentative, and each mark would explain itself through the triggering pair.
• I3 (tree): when gparse or the perceptron parser cannot build a confident tree, per-token valency and category predictions could be kept as a fallback layer for later stages.
• I4 (world): the scope of a hedge or negation could be projected along UD paths, so that only the states and events inside it become uncertain in the world model. A learned path-compression step could be tested as a self-induced rule.
• I5 (learn): run a lexicon-only baseline as a control for each new classifier, and use sparse priors to keep feature sets small enough to inspect.