Elamur Research⌕
☰
home / papers / 2010 / Automatic Committed Belief Tagging

Automatic Committed Belief Tagging

Paper · 2010 · Vinodkumar Prabhakaran, Owen Rambow and Mona Diab

Summary

The paper moves past extracting bare propositions and asks a further question: which propositions does the writer present as their own firm belief, which as tentative or merely reported, and which as having no truth value at all (wishes, questions, demands)? The authors use a 10,000-word multi-genre corpus (Diab et al., 2009) in which each propositional head carries one of three labels: committed belief (CB), non-committed belief (NCB) or not applicable (NA). Every other token is labeled O. They treat the problem as one joint four-way token-tagging task, so locating heads and classifying them happen in a single step. They compare SVM sequence labeling (YAMCHA) with linear-chain CRF (MALLET). Features fall into two groups: lexical ones, and syntactic ones read off the MICA deep dependency parser. With the best feature combination the system reaches 64% F-measure. Adding syntax cuts F-measure error by 21% relative to a lexical-only setup, which shows that automatic parses help even though they contain errors. The work points toward representing text as a record of cognitive states rather than as a flat list of facts.

Key points

  • Annotation scheme: CB covers firmly believed propositions, past or future. NCB covers epistemic hedging ("may") and reported speech. NA covers desires, questions and requirements. Labels follow the perceived intention of the writer, not surface form, so a rhetorical question can be tagged CB.
  • Event nominals receive no belief annotation in this corpus and are always tagged O.
  • Useful syntactic features from MICA include:
    • an ancestor that is a reporting or cognition verb (say, claim, think, doubt, …);
    • perfect-auxiliary, "should", or wh-word daughters;
    • the identity of any auxiliary or modal daughter;
    • the parent's POS.
  • Several candidates were discarded after an exhaustive, incremental pruning search: lemma, Porter stem, supertags, isRoot, and pred.
  • YAMCHA settings: one-versus-all multiclass with a quadratic kernel and c = 0.5. Context windows ranged from 0 to 5 tokens, and syntactic features were tested both with and without context. A pipeline variant was also tried, and its results supported choosing the joint model.

Interesting ideas

  • I1. Each clause head is labeled for how strongly the writer commits to it, as a dimension separate from what the clause says; reported speech is kept apart from wishes and questions, which carry nobody's belief.
  • I2. Instead of segmenting text into propositions, the system tags only proposition heads, which correspond one-to-one with propositions, and so it avoids a hard segmentation step.
  • I3. A cheap tree feature checks whether any ancestor is a reporting or cognition verb from a short lemma list, and it captures embedded attribution well.
  • I4. Sentence-level accuracy correlated weakly with parser confidence, and the authors propose using that confidence to choose sentences for self-training.
  • I5. Epistemic and deontic modals share the same syntax but call for different labels, and the authors flag this as a known failure mode.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): Each event or state in the text world could store a commitment flag (CB/NCB/NA), so that wishes and questions never enter the world as facts.
  • I2 (frames): Commitment labels can sit directly on the predicate node of each frame in the UD tree, with no separate clause splitter needed.
  • I3 (frames): An explainable rule could walk up UD heads from a clause and mark it as attributed to the subject of a say/claim/think verb, not to the author.
  • I4 (learn): Silver data for downstream taggers could be filtered by parse confidence or by witness-parser agreement, so that noisy syntax features are kept out.
  • I5 (answer): Test sets could include contrasting pairs such as "may" or "must" in the epistemic versus deontic sense, used as targeted negative controls to check whether readers distinguish possibility from obligation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Columbia UniversityNew York · United States149 works · 48,852 citations

📚 Shelves 4 Computer Science · 9,086 works Artificial Intelligence · 8,889 works Arts and Humanities · 9,296 works Language and Linguistics · 9,206 works

Cite

Cite as Vinodkumar Prabhakaran, Owen Rambow and Mona Diab. 2010. Automatic Committed Belief Tagging. Coling 2010: Posters.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{prabhakaran2010automatic,
  title = {{Automatic Committed Belief Tagging}},
  author = {Vinodkumar Prabhakaran, Owen Rambow and Mona Diab},
  booktitle = {Coling 2010: Posters},
  year = {2010},
  url = {https://aclanthology.org/C10-2117/}
}

References

24 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bangalore, Srinivas and Aravind Joshi. 1999. Supertagging: An approach to almost parsing. Computational Linguistics, 25(2):237–266. our card
  2. Bangalore, Srinivas, Pierre Boullier, Alexis Nasr, Owen Rambow, and Benoı̂t Sagot. 2009. MICA: A probabilistic dependency parser based on tree insertion grammars. In NAACL HLT 2009 (Short Papers). find
  3. Bratman, Michael E. 1999 [1987]. Intention, Plans, and Practical Reason. CSLI Publications. find
  4. Bunt, Harry. 2000. Dialogue pragmatics and context specification. In Bunt, Harry and William J. Black, editors, Abduction, Belief and Context in Dialogue, pages 81–150. ↗
  5. Clark, Herbert H. 1996. Using Language. cup, Cambridge, England. find
  6. Cohen, Philip R. and Hector J. Levesque. 1990. Rational interaction as the basis for communication. In Philip Cohen, Jerry Morgan and James Allen, editors, Intentions in Communication. MIT Press. ↗
  7. Diab, Mona T., Lori Levin, Teruko Mitamura, Owen Rambow, Vinodkumar Prabhakaran, and Weiwei Guo. 2009. Committed belief annotation and tagging. In ACL-IJCNLP ’09: Proceedings of the Third Linguistic Annotation Workshop, pages 68– 73, Morristown, NJ, USA. Association for Computational Linguistics. find
  8. Grice, Herbert Paul. 1975. Logic and conversation. In Cole, P. and J. Morgan, editors, Syntax and semantics, vol 3. Academic Press, New York. ↗
  9. Hovy, Eduard H. 1993. Automated discourse generation using discourse structure relations. Artificial Intelligence, 63:341–385. ↗
  10. Krestel, Ralf, Sabine Bergler, and René Witte. 2008. Minding the Source: Automatic Tagging of Reported Speech in Newspaper Articles. In (ELRA), European Language Resources Association, editor, Proceedings of the Sixth International Language Resources and Evaluation (LREC 2008), Marrakech, Morocco, May 28–30. find
  11. Kudo, Taku and Yuji Matsumoto. 2000. Use of support vector learning for chunk identification. In Proceedings of CoNLL-2000 and LLL-2000, pages 142–144. find
  12. Mann, William C. and Sandra A. Thompson. 1987. Rhetorical Structure Theory: A theory of text organization. Technical Report ISI/RS-87-190, ISI. find
  13. McArthur, Gregory L. 1988. Reasoning about knowledge and belief: a survey. Computational Intelligence, 4:223–243. ↗
  14. McCallum, Andrew Kachites. 2002. Mallet: A machine learning for language toolkit. http://mallet.cs.umass.edu. ↗
  15. Moore, Johanna. 1994. Participating in Explanatory Dialogues. MIT Press. ↗
  16. Perrault, C. Raymond and James F. Allen. 1980. A plan-based analysis of indirect speech acts. Computational Linguistics, 6(3–4):167–182. ↗ our card
  17. Rao, Anand S. and Michael P. Georgeff. 1991. Modeling rational agents within a BDI-architecture. In Allen, James, Richard Fikes, and Erik Sandewall, editors, Proceedings of the 2nd International Conference on Principles of Knowledge Representation and Reasoning, pages 473–484. Morgan Kaufmann publishers Inc.: San Mateo, CA, USA. find
  18. Saurı́, Roser and James Pustejovsky. 2007. Determining Modality and Factuality for Textual Entailment. In First IEEE International Conference on Semantic Computing., Irvine, California. find
  19. Saurı́, Roser and James Pustejovsky. 2008. From Structure to Interpretation: A Double-layered Annotation for Event Factuality. In Proceedings of the 2nd Linguistic Annotation Workshop. LREC 2008. find
  20. Somasundaran, Swapna, Janyce Wiebe, and Josef Ruppenhofer. 2008. Discourse level opinion interpretation. In Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 801–808, Manchester, UK, August. Coling 2008 Organizing Committee. find
  21. Stolcke, Andreas, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Daniel Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational Linguistics, 26:339–373. ↗ our card
  22. Stone, Matthew. 2004. Intention, interpretation and the computational structure of language. Cognitive Science, 24:781–809. ↗
  23. Wiebe, Janyce, Theresa Wilson, Rebecca Bruce, Matthew Bell, and Melanie Martin. 2004. Learning subjective language. In Computational Linguistics, Volume 30 (3). ↗ our card
  24. Wilks, Yorick and Afzal Ballim. 1987. Multiple agents and the heuristic ascription of belief. In Proceedings of the 10th International Joint Conference on Artificial Intelligence (IJCAI), pages 118–124. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/prabhakaran-2010-automatic-committed-belief-tagging/