Elamur Research
☰
home / papers / 2018 / Using Hedge Detection to Improve Committed Belief Tagging

Using Hedge Detection to Improve Committed Belief Tagging

What it is about

The paper adds hedge features to a committed-belief tagger. The tagger marks each proposition head as committed belief (CB), non-committed belief (NCB), reported belief (ROB) or non-belief (NA: desire, intention, question). Hedges are found from a 117-term dictionary in two ways: plain lemma lookup, or hand rules over POS and dependencies that separate hedge from non-hedge uses. For example, about as a preposition is not a hedge; likely as amod of a noun is not; assume, tend and appear are hedges only with a clausal complement; sure and completely are hedges only under negation; supposed to and should have are not. Features for the word itself, its parent, children, siblings and ancestors, plus a sentence flag, go into an SVM. On the DEFT forum corpus (5-fold CV), rule-based hedge features raise NCB F1 from 23.7 to 30.4 and ROB from 23.3 to 27.4; overall F1 rises only from 67.5 to 68.1, because those classes are rare.

Ideas

For Mova

languages/en/32-politeness-and-hedging.md §8 (epistemic modality and evidentiality), languages/en/33-implicature-and-presupposition.md (speaker commitment); prag/src/frames.rs, prag/src/feats.rs, world/src/knowledge.rs; measures: frame bench F1 unchanged (no regression), course test counts, the diagnose breakdown.

Library summary

The Research library's own summary of the paper, in our words — not the paper's text.

Written for the Research library (claude-opus-5-5) from the paper text. Original — aclanthology.org/W18-1301.pdf

Summary

The paper asks whether recognizing hedges, meaning expressions that signal uncertainty or distance from a claim, can help a committed belief tagger. Such a tagger judges how strongly a writer stands behind each proposition. Many hedge cues are ambiguous: "around" can express approximation or physical direction. No hedge-annotated corpus of informal text was available, so the authors avoided supervised learning. They built a 117-entry dictionary of potential hedges, divided into relational and propositional types. They then wrote hand-crafted disambiguation rules for selected entries, using context, part-of-speech tags and dependency relations. Hedge information was fed into an existing SVM-based belief tagger as features at three levels: the word itself, its dependency neighbours, and the whole sentence. The authors compared three setups on discussion forum data: no hedge features, dictionary lookup, and rule-based detection. Both hedge-aware setups improved results, and the rule-based one helped most. The gains were concentrated in the rare reported-belief and non-committed-belief classes. The work shows that lightweight, annotation-free hedge detection can benefit belief and factuality analysis in new genres.

Key points

  • Hedge dictionary: 117 terms, seeded from CoNLL-2010 hedge cues and WordNet synonyms, then refined with input from the LDC and other linguists. Entries are split into relational hedges (e.g., think) and propositional hedges (e.g., sort of).
  • Tagger and evaluation: The tagger is the quadratic-kernel SVM of Prabhakaran et al. (2010), also known as System C in Prabhakaran et al. (2015). It labels proposition heads as CB, NCB, ROB or NA. Evaluation used 5-fold cross-validation over the committed belief corpus released by DARPA DEFT in 2014 (LDC2014E55).
  • F-measure gains (baseline → dictionary → rules):
    • ROB rose from 23.29 to 25.11 to 27.43.
    • NCB rose from 23.66 to 25.95 to 30.43, a gain of 6.77 with the rules.
  • Overall F-measure: 67.52 → 67.95 → 68.07. The paper's prose gives 69.95 for the dictionary setup, but the results table shows 67.95.
  • Future work: The authors are building a forum-post hedge corpus via Amazon Mechanical Turk. Workers choose between hedging and non-hedging definitions of each highlighted term. The corpus will be used to evaluate the rules and to train machine learning hedge classifiers.

Cite

The paper itself, not this card. BibTeX from the library's record of the work. DOI 10.18653/v1/w18-1301

@inproceedings{ulinski2018using,
  title = {{Using Hedge Detection to Improve Committed Belief Tagging}},
  author = {Ulinski, Morgan and Benjamin, Seth and Hirschberg, Julia},
  booktitle = {Proceedings of the Workshop on Computational Semantics beyond Events and Roles},
  year = {2018},
  doi = {10.18653/v1/w18-1301},
  url = {https://doi.org/10.18653/v1/w18-1301}
}

References

4 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Veronika Vincze, György Szarvas, Richárd Farkas, György Móra, and János Csirik. 2008. The bioscope corpus: biomedical texts annotated for uncertainty, negation and their scopes. BMC Bioinformatics, 9(11):S9. find
  2. David Clausen. 2010. Hedgehunter: A system for hedge detection and uncertainty classification. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 120– 125, Uppsala, Sweden. Association for Computational Linguistics. find
  3. Andreas Vlachos and Mark Craven. 2010. Detecting speculative language using syntactic dependencies and logistic regression. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 18–25, Uppsala, Sweden. Association for Computational Linguistics. find
  4. Richárd Farkas, Veronika Vincze, György Móra, János Csirik, and György Szarvas. 2010. The conll-2010 shared task: Learning to detect hedges and their scope in natural language text. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 1–12, Uppsala, Sweden. Association for Computational Linguistics. Vinodkumar Prabhakaran, Tomas By, Julia Hirschberg, Owen Rambow, Samira Shaikh, Tomek Strzalkowski, Jennifer Tracey, Michael Arrigo, Ru-payan Basu, Micah Clark, Adam Dalton, Mona find our card