Elamur Research⌕
☰
home / papers / 2013 / A Discriminative Analysis of Fine-Grained Semantic Relations including Presupposition: Annotation and Classification

A Discriminative Analysis of Fine-Grained Semantic Relations including Presupposition: Annotation and Classification

Paper · 2013 · Galina Tremper and Anette Frank

Summary

Verbs often carry hidden meaning. Saying someone won a match implies they played it, even when the sentence is negated. Such lexically triggered presuppositions are largely absent from resources like WordNet and had not previously been targeted by corpus-based relation learning. Tremper and Frank treat the acquisition of presupposing verb pairs as a discriminative classification problem. Presupposition is separated from entailment, temporal inclusion, antonymy and an unrelated class. The authors first analyse the logical properties that set these relations apart, especially temporal order and behaviour under negation. They turn these properties into type-level corpus features and add features for surface contiguity and semantic relatedness. Conventional annotation proved unreliable, so they designed a question-based scheme over generated skeleton sentences, which produced a usable gold standard. With roughly ten training verb pairs per relation, the best (hierarchical) classifier reaches 55% F1, about double a best-feature baseline. The work shows that both manual and automatic discrimination of these subtle inferential relations is feasible, which is a step toward enriching discourse representations with implicit content.

Key points

  • The relation inventory covers presupposition, entailment, temporal inclusion, antonymy and an unrelated class. The negation test is the core criterion separating presupposition, which survives negation, from entailment, which does not.
  • The annotation design gives verb pairs prototypical arguments drawn from selectional preference classes and automatically builds skeleton sentences. Complex judgments are split into simple questions. An attempt to hand the task to a non-expert did not succeed, so it is not yet ready for crowd-sourcing.
  • Two classifier designs were compared, flat and hierarchical. The hierarchical one leads by about 4 F1 points, but the gap is not significant. Both clearly beat the best single-feature baseline, by up to 100%, with an overall score of 55% F1.
  • Antonymy is the easiest relation to classify. Presupposition is classified with higher precision than entailment, likely because of its distinctive negation behaviour.
  • Feature impact differs by relation:
    • temporal features help most for presupposition, entailment and temporal inclusion;
    • relatedness features help most for antonymy and the unrelated class;
    • negation features matter most for presupposition.
  • The main error sources are verb sense ambiguity, argument coreference, preprocessing noise and error propagation in the hierarchical setup.

Interesting ideas

  • I1. Use behaviour under negation as an operational test: presupposed content still holds when the trigger is negated, while entailed content requires the trigger to be outside negation scope. This gives a clean, checkable rule for which inferences are licensed.
  • I2. Derive classification features directly from the logical signature of each target relation, such as temporal order between the two events and negation behaviour, instead of relying on generic distributional similarity, which cannot separate fine relations.
  • I3. Break a hard holistic judgment into a sequence of small yes/no questions posed over automatically generated skeleton sentences with prototypical arguments. This controls verb sense and removes distracting context.
  • I4. Arrange classification coarse-to-fine in a hierarchy, accepting that errors can propagate between levels and noting collective classification as a possible remedy.
  • I5. Report feature impact per relation rather than only overall scores, to reveal which evidence drives which category.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (world): When a trigger verb like win or lift appears, add its presupposed event (played, had been imposed) to the world even under negation, but add entailments (killed → died) only for non-negated triggers.
  • I2 (world): Store verb-pair relations in compiled knowledge with explicit attributes (temporal precedence/inclusion, negation-robustness), so every derived inference can be explained by the attribute that licensed it.
  • I3 (learn): LLM teacher tiers could be asked such decomposed, templated questions about verb pairs instead of one complex labelling question, making silver labels more consistent and auditable.
  • I4 (frames): Implicature and inference typing could first decide whether a verb pair is related at all and only then pick the fine relation, with each level's decision logged for explainability.
  • I5 (learn): Evaluation of inference rules could include per-relation ablations, for example removing negation cues, as a diagnostic alongside negative controls.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Heidelberg UniversityHeidelberg · Germany27 works · 40,944 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.Implicature · 72 works

Cite

Cite as Galina Tremper and Anette Frank. 2013. A Discriminative Analysis of Fine-Grained Semantic Relations including Presupposition: Annotation and Classification. Dialogue & Discourse Volume 4.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.5087/dad.2013.212

@inproceedings{tremper2013discriminative,
  title = {{A Discriminative Analysis of Fine-Grained Semantic Relations including Presupposition: Annotation and Classification}},
  author = {Galina Tremper and Anette Frank},
  booktitle = {Dialogue \& Discourse Volume 4},
  year = {2013},
  doi = {10.5087/dad.2013.212},
  url = {https://doi.org/10.5087/dad.2013.212}
}

References

49 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ben Roni Aharon, Idan Szpektor, and Ido Dagan. Generating entailment rules from FrameNet. In Proceedings of the ACL 2010 Conference Short Papers, pages 241–246, Uppsala, Sweden, 2010.
  2. James F. Allen. Maintaining knowledge about temporal intervals. In Communications of the ACM, pages 832–843. ACM Press, November 1983. ↗
  3. Marco Baroni and Alessandro Lenci. How we BLESSed distributional semantic evaluation. In Proceedings of the GEMS 2011 Workshop on GEometrical Models of Natural Language Semantics, pages 1–10, Edinburgh, UK, 2011.
  4. Marco Baroni, Silvia Bernardini, Adriano Ferraresi, and Eros Zanchetta. The WaCky Wide Web: a collection of very large linguistically processed web-crawled corpora. In Journal of Language Resources and Evaluation, volume 43(3), pages 209–226, 2009. ↗
  5. Jonathan Berant, Ido Dagan, and Jacob Goldberger. Global learning of focused entailment graphs. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL 2010), pages 1220–1229, Uppsala, Sweden, 2010. find
  6. Jonathan Berant, Ido Dagan, and Jacob Goldberger. Learning entailment relations by global graph structure optimization. Computational Linguistics, 38(1):73–111, 2012. ↗
  7. Steven Bethard and James H. Martin. CU-TMP: Temporal relation classification using syntactic and semantic features. In Proceedings of the 4th International Workshop on Semantic Evaluations, SemEval ’07, pages 129–132, Prague, Czech Republic, 2007. find
  8. Johan Bos. Implementing the binding and accommodation theory for anaphora resolution and presupposition projection. Computational Linguistics, 29(2):179–210, 2003. ↗
  9. Nathanael Chambers and Dan Jurafsky. Unsupervised learning of narrative event chains. In Proceedings of the ACL/HLT 2008 Conference, pages 789–797, Columbus, Ohio, 2008. find
  10. Nathanael Chambers and Dan Jurafsky. Unsupervised learning of narrative schemas and their participants. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 602–610, Suntec, Singapore, 2009. find
  11. Nathanael Chambers, Shan Wang, and Dan Jurafsky. Classifying temporal relations between events. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics (ACL 2007), Prague, Czech Republic, 2007. find
  12. Timothy Chklovski and Patrick Pantel. VerbOcean: Mining the web for fine-grained semantic verb relations. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pages 33–40, Barcelona, Spain, 2004. find
  13. David R. Clausen and Christopher D. Manning. Presupposed content and entailments in Natural Language Inference. In Proceedings of the 2009 Workshop on Applied Textual Inference, ACL-IJCNLP 2009, pages 70–73, Suntec, Singapore, 2009. find
  14. Jacob A. Cohen. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, pages 37–46, 1960. ↗
  15. Ido Dagan, Bill Dolan, Bernardo Magnini, and Dan Roth. Recognizing textual entailment: Rational, evaluation and approaches. Natural Language Engineering, 15(Special Issue 04):i–xvii, 2009. ↗
  16. Christian Fellbaum, editor. WordNet: An Electronic Lexical Database. MIT Press, 1998. find
  17. Charles J. Fillmore, Christopher R. Johnson, and Miriam R. L. Petruck. Background to Framenet. International Journal of Lexicography, 16(3):235–250, 2003. ↗
  18. John R. Firth. A synopsis of linguistic theory 1930–1955. Studies in linguistic analysis, pages 1–32, 1957. find
  19. Anette Frank and Sebastian Pádo. Semantics in computational lexicons. In Claudia Maienborn, Klaus Heusinger, and Paul Portner, editors, Semantics: An International Handbook of Natural Language Meaning, volume 3 of HSK Handbooks of Linguistics and Communication Science Series, pages 2887–2917. Mouton de Gruyter, 2012. find
  20. L. T. F. Gamut. Logic, Language, and Meaning: Introduction to Logic. University of Chicago Press, Chicago, 1991. find
  21. Michael L. Geis and Arnold M. Zwicky. On invited inferences. Linguistic Inquiry, 2(4):561–566, 1971. find
  22. Bart Geurts and David Beaver. Presuppostion. In Claudia Maienborn, Klaus Heusinger, and Paul Portner, editors, Semantics: An International Handbook of Natural Language Meaning, HSK Handbooks of Linguistics and Communication Science Series. Mouton de Gruyter, 2012. find
  23. Roxana Girju, Adriana Badulescu, and Dan Moldovan. Automatic discovery of part-whole relations. Computational Linguistics, 32:83–135, 2006. ↗
  24. Marti Hearst. Automatic acquisition of hyponyms from large text corpora. In Proceedings of the Fourteenth International Conference on Computational Linguistics (COLING), pages 539–545, Nantes, France, 1992. find
  25. Verena Henrich and Erhard Hinrichs. Standardizing Wordnets in the ISO Standard LMF: Wordnet-LMF for GermaNet. In Proceedings of the 23rd International Conference on Computational Linguistics (COLING), pages 456–464, Beijing, China, 2010. ↗
  26. Takashi Inui, Kentaro Inui, and Yuji Matsumoto. Acquiring causal knowledge from text using the connective marker tame. ACM Transactions on Asian Language Information Processing (TALIP), 4(4):435–474, 2005. ↗
  27. Hans Kamp and Uwe Reyle. From Discourse to Logic. Introduction to Modeltheoretic Semantics of Natural Language, Formal Logic and Discourse Representation Theory. Kluwer, Dordrecht, 1993. ↗
  28. Lauri Karttunen. Simple and phrasal implicatives. In Proceedings of *SEM: The First Joint Conference on Lexical and Computational Semantics, pages 124–131, Montréal, Canada, 2012. find
  29. Claudia Kunze and Lothar Lemnitzer. Germanet - representation, visualization, application. In Proceedings of the 3rd International Language Resources and Evaluation Conference (LREC’02), pages 1485–1491, Las Palmas, Canary Islands, 2002. find
  30. Chong Min Lee. Temporal relation identification with endpoints. In Proceedings of the NAACL/HLT 2010 Student Research Workshop, pages 40–45, Los Angeles, California, 2010. find
  31. Heeyoung Lee, Yves Peirsman, Angel Chang, Nathanael Chambers, Mihai Surdeanu, and Dan Jurafsky. Stanford’s multi-pass sieve coreference resolution system at the CoNLL-2011 shared task. In Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task, pages 28–34, Portland, Oregon, USA, 2011.
  32. Stephen C. Levinson. Pragmatics. Cambridge: Cambridge University Press, 1983. find
  33. Dekang Lin and Patrick Pantel. Discovery of inference rules for Question Answering. Natural Language Engineering, 7:343–360, 2001. ↗
  34. John Lyons. Semantics. Cambridge University Press, 1977. find
  35. Marie-Catherine de Marneffe, Bill MacCartney, and Christopher D. Manning. Generating typed dependency parses from phrase structure parses. In In Proc. Intl Conf. on Language Resources and Evaluation (LREC, pages 449–454, 2006.
  36. Saif Mohammad and Graeme Hirst. Distributional measures of semantic distance: A survey. arXiv:1203.1858v1, 2012. First published in 2006. ↗
  37. Robert Munro, Steven Bethard, Victor Kuperman, Vicky Tzuyin Lai, Robin Melnick, Christopher Potts, Tyler Schnoebelen, and Harry Tily. Crowdsourcing and language studies: the new generation of linguistic data. In Proceedings of the NAACL/HLT 2010 Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk, pages 122–130, Los Angeles, California, 2010.
  38. Patrick Pantel and Marco Pennacchiotti. Espresso: Leveraging generic patterns for automatically harvesting semantic relations. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 113–120, Sydney, Australia, 2006.
  39. Viktor Pekar. Discovery of event entailment knowledge from text corpora. Computer Speech & Language, 22(1):1–16, 2008. ↗
  40. Michaela Regneri, Alexander Koller, and Manfred Pinkal. Learning script knowledge with web experiments. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL 2010), pages 979–988, Uppsala, Sweden, 2010. find
  41. Philip Resnik. Selectional constraints: An information-theoretic model and its computational realization. Cognition, 61:127–159, 1996. ↗
  42. Tom Richens. Anomalies in the WordNet verb hierarchy. In Proceedings of the 22nd International Conference on Computational Linguistics (COLING), pages 729–736, Manchester, UK, 2008. find
  43. Rob van der Sandt. Presupposition projection as anaphora resolution. Journal of Semantics, 9: 333–377, 1992. ↗
  44. Peter F. Strawson. On referring. Mind, 59(235):320–344, 1950. find
  45. Galina Tremper. Weakly supervised learning of presupposition relations between verbs. In Proceedings of the ACL 2010 Student Research Workshop, pages 97–102, Uppsala, Sweden, 2010.
  46. Galina Tremper and Anette Frank. Extending semantic relation classification to presupposition relations between verbs. In Proceedings of the DGfS Workshop: “Beyond Semantics: Corpus-based investigations of pragmatic and discourse phenomena”, Göttingen, Germany, 2011. find
  47. Marc Verhagen, Roser Sauri, Tommaso Caselli, and James Pustejovsky. Semeval-2010 task 13: Tempeval-2. In Proceedings of the 5th International Workshop on Semantic Evaluation, pages 57–62, Uppsala, Sweden, 2010. find
  48. Hila Weisman, Jonathan Berant, Idan Szpektor, and Ido Dagan. Learning verb inference rules from linguistically-motivated evidence. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, pages 194–204, Jeju Island, Korea, 2012. find
  49. Ian H. Witten and Eibe Frank. Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann Publishers, 2nd edition, 2005. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/tremper-2013-discriminative-analysis-fine-grained-semantic/