Elamur Research⌕
☰
home / papers / 2010 / Exploiting CCG Structures with Tree Kernels for Speculation Detection

Exploiting CCG Structures with Tree Kernels for Speculation Detection

Paper · 2010 · Liliana Mamani Sánchez, Baoli Li and Carl Vogel

Summary

This paper addresses Task 1 of the CoNLL-2010 Shared Task: deciding whether a sentence contains uncertainty (hedging). The authors treat it as disambiguating candidate cue words, because a cue such as "may" does not always mark speculation.

The pipeline works in several steps:

  • A cue lexicon, including multi-word cues, is harvested from the training data with frequencies. It tags candidate cues using maximal matching, so longer phrases win.
  • Sentences are tokenized and lemmatized with GeniaTagger and parsed with the C&C Parser into CCG derivations.
  • For each candidate cue, a local subtree around a token window is fed to an SVM tree-kernel classifier.
  • Sentences without candidate cues fall to a keyword baseline or a maximum entropy bag-of-features model. That model uses n-grams of words, lemmas, POS and supertags plus document-position features.

The system was evaluated on Wikipedia and BioScope data. Its official Wikipedia in-domain result was about 0.55 F. On biomedical data, the CCG-based classifiers clearly beat the keyword baseline. The work suggests that CCG structure helps separate genuine hedges from apparent ones.

Key points

  • Datasets are in BioScope XML format. The biomedical set has 14,541 training and 5003 evaluation sentences, and the Wikipedia set has 11,111 training and 9634 evaluation sentences.
  • The official submission covered only the Wikipedia domain, reaching about 0.55 F in-domain. The keyword-only baseline scored 0.5689 F on Wikipedia.
  • On biomedical data, the keyword baseline has near-total recall (0.9949) but precision of only 0.2261, giving 0.3685 F. This shows that cue presence alone is insufficient.
  • The tree kernel follows Moschitti (2006) and uses subset trees (SST) rather than plain subtrees, so internal non-leaf structure is compared. The maximum entropy model follows Zhang (2004).
  • An earlier Wikipedia hedge detector (Ganter & Strube, 2009) reported 0.69 F, but it was evaluated on different Wikipedia data.

Interesting ideas

  • I1. Treat hedge detection as word-sense disambiguation of lexicon cues: first over-generate candidates from a harvested lexicon, then let a classifier decide per occurrence whether the cue really hedges.
  • I2. Resolve overlapping cue matches by maximal matching, so specific multi-word phrases like "the most common" take priority over a bare "most".
  • I3. Cut the parse to a local fragment: take an n-token window around the cue, root it at the lowest node covering the window, and collapse extra out-of-window tokens into one placeholder symbol.
  • I4. When two cues' windows yield the same root, emit one example; otherwise emit separate ones, avoiding duplicated training instances.
  • I5. Back the structured classifier with a fallback for uncovered sentences, and keep a trivial keyword baseline as a reference whose high-recall, low-precision profile exposes how much the structure actually adds.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): The prag crate could flag modal and hedge cues from a lexicon and then confirm each one from its tree context before marking a proposition as uncertain rather than asserted.
  • I2 (tokens): A multi-word expression tagger in crate en could apply longest-match priority so that downstream frames see one unit instead of competing fragments.
  • I3 (tree): Local, normalized tree fragments around a target word could serve as compact, explainable features or repair-rule patterns instead of whole-sentence trees.
  • I4 (learn): When building silver training examples around multiple triggers in a sentence, deduplicating by shared subtree root would keep the data from overweighting dense sentences.
  • I5 (learn): Evaluations of uncertainty or modality marking could always report a lexicon-only baseline as a negative control, so gains from tree-based reasoning are measured honestly.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Trinity College DublinDublin · Ireland17 works · 1,222 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Syntax & grammar formalisms · 161 works

Cite

Cite as Liliana Mamani Sánchez, Baoli Li and Carl Vogel. 2010. Exploiting CCG Structures with Tree Kernels for Speculation Detection. Proceedings of the Fourteenth Conference on Computational Natural Language Learning – Shared Task.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{sanchez2010exploiting,
  title = {{Exploiting CCG Structures with Tree Kernels for Speculation Detection}},
  author = {Liliana Mamani Sánchez, Baoli Li and Carl Vogel},
  booktitle = {Proceedings of the Fourteenth Conference on Computational Natural Language Learning – Shared Task},
  year = {2010},
  url = {https://aclanthology.org/W10-3018/}
}

References

15 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aristotle. trans. 1991. The Art of Rhetoric. Penguin Classics, London. Translated with an Introduction and Notes by H.C. Lawson-Tancred. find
  2. Stephen Boxwell, Dennis Mehay, and Chris Brew. 2009. Brutus: A semantic role labeling system incorporating CCG, CFG, and dependency features. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 37–45, Suntec, Singapore. find
  3. Ted Briscoe and John Carroll. 2006. Evaluating the accuracy of an unlexicalized statistical parser on the PARC DepBank. In Proceedings of the COL-ING/ACL on Main conference poster sessions, pages 41–48, Morristown, NJ, USA. find
  4. Stephen Clark and James R. Curran. 2004. Parsing the WSJ using CCG and log-linear models. In ACL ’04: Proceedings of the 42nd Annual Meeting on Association for Computational Linguistics, page 103, Morristown, NJ, USA.
  5. Richárd Farkas, Veronika Vincze, György Móra, János Csirik, and György Szarvas. 2010. The CoNLL-2010 Shared Task: Learning to Detect Hedges and their Scope in Natural Language Text. In Proceedings of the Fourteenth Conference on Computational Natural Language Learning (CoNLL-2010): Shared Task, pages 1–12, Uppsala, Sweden, July. Association for Computational Linguistics.
  6. Viola Ganter and Michael Strube. 2009. Finding hedges by chasing weasels: Hedge detection using Wikipedia tags and shallow linguistic features. In Proceedings of the ACL-IJCNLP 2009 Conference Short Papers, pages 173–176, Suntec, Singapore.
  7. Daniel Gildea and Julia Hockenmaier. 2003. Identifying semantic roles using combinatory categorial grammar. In Proceedings of 2003 Conference on Empirical Methods in Natural Language Processing (EMNLP), Sapporo, Japan. ↗
  8. George Lakoff. 1973. Hedges: A study in meaning criteria and the logic of fuzzy concepts. Journal of Philosophical Logic, 2(4):458–508. ↗
  9. Ben Medlock and Ted Briscoe. 2007. Weakly supervised learning for hedge classification in scientific literature. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 992–999, Prague, Czech Republic.
  10. Alessandro Moschitti. 2006. Making tree kernels practical for natural language learning. In Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics. find
  11. Arzucan Özgür and Dragomir R. Radev. 2009. Detecting speculations and their scopes in scientific text. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 1398–1407, Singapore. find
  12. Mark Steedman. 2000. The syntactic process. MIT Press, Cambridge, MA, USA. ↗
  13. György Szarvas, Veronika Vincze, Richárd Farkas, and János Csirik. 2008. The BioScope corpus: annotation for negation, uncertainty and their scope in biomedical texts. In Proceedings of the Workshop on Current Trends in Biomedical Natural Language Processing, pages 38–45, Columbus, Ohio.
  14. Yoshimasa Tsuruoka, Yuka Tateishi, Jin-Dong Kim, Tomoko Ohta, John McNaught, Sophia Ananiadou, and Jun’ichi Tsujii. 2005. Developing a robust part-of-speech tagger for biomedical text. In Advances in Informatics, pages 382–392. find
  15. Le Zhang. 2004. Maximum entropy modeling toolkit for Python and C++ (version 20041229). In Natural Language Processing Lab, Northeastern. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sanchez-2010-exploiting-ccg-structures-tree-kernels/