Elamur Research⌕
☰
home / papers / 2020 / The Universal Decompositional Semantics Dataset and Decomp Toolkit

The Universal Decompositional Semantics Dataset and Decomp Toolkit

Paper · 2020 · Aaron Steven White, Elias Stengel-Eskin, Siddharth Vashishtha et al.

Summary

Older semantic annotation schemes rely on large, mutually exclusive category inventories. They need expert annotators and fail on borderline cases, on items that fit several categories, and on phenomena the ontology left out. Decompositional semantics replaces categories with many simple questions that untrained crowd workers can answer, but earlier datasets of this kind were released separately with no common interface. This paper presents UDS1.0, which merges five such annotation sets into one graph format, and Decomp0.1, a Python 3 toolkit for querying the graphs.

The graphs sit on top of the English Web Treebank and have three layers:

  • gold UD-EWT dependency trees,
  • predicate–argument structures that PredPatt extracts deterministically from those trees,
  • real-valued semantic attributes with confidence scores, derived from normalized crowd judgments.

Users can query everything with SPARQL 1.1. The result is a single public resource (decomp.io) for fine-grained semantic parsing that can grow by adding attributes rather than redesigning the scheme.

Key points

  • UDS1.0 combines five decompositional annotation sets:
    • semantic proto-roles,
    • entity types,
    • event factuality,
    • genericity of statements about entities and events,
    • temporal properties of events and relations between them.
  • The base layer is gold UD-EWT (v1.2) in CoNLL-U, and UDS1.0 keeps that treebank's train/dev/test splits. Token nodes carry form, lemma, upos, xpos and UD features, and edges carry deprel.
  • Graph elements are labeled with one of three domains:
    • syntax,
    • semantics,
    • interface, for instance edges of type head or nonhead that link predicates and arguments to their tokens.
  • A clausal complement appears as an argument node with a semantics-domain head edge to the embedded predicate.
  • Every graph also contains four performative nodes that have no attributes yet:
    • an argument node for the whole sentence,
    • a predicate node for the author producing the sentence for the addressee,
    • an argument node for the author,
    • an argument node for the addressee.
  • Decomp0.1 is a Python 3 suite for SPARQL queries and RDF-style serialization of the graphs.

Interesting ideas

  • I1. Semantic meaning is stored as a bundle of independent graded properties, each answering one simple question, rather than a single category label; new properties can be added later without reannotating old data.
  • I2. Every semantic node keeps explicit links to its syntactic head and to the other tokens in its span, and these links form a separate interface domain. This keeps each meaning element traceable to the exact words it came from.
  • I3. The whole utterance is wrapped in placeholder nodes for the speaker, the hearer and the act of saying. This leaves room for pragmatic annotation later without changing the graph structure.
  • I4. Attributes are real numbers paired with confidence scores, not hard yes/no values, so a downstream consumer can tell weak evidence from strong.
  • I5. The unified graph is exposed through a standard query language (SPARQL), so users ask questions of the data instead of writing ad hoc traversal code.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): Alongside each VerbNet role label, keep proto-role-style properties such as volition or change of state, so that unusual arguments still get a sensible, explainable description.
  • I2 (tree): Mark the UD→frame links as head or nonhead edges in their own layer, so every frame argument can point back to the tokens that justify it.
  • I3 (frames): Give each sentence's frame graph an author, addressee and utterance predicate, so that speech acts, implicature and irony can attach to them.
  • I4 (learn): Store agreement among the witness parsers or LLM teacher tiers as a confidence value on each silver attribute, and weight training or filtering by it.
  • I5 (world): A declarative query layer over the world graph would let readers and tests retrieve entities, states and event order in a transparent, inspectable form.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

Cite

Cite as Aaron Steven White, Elias Stengel-Eskin, Siddharth Vashishtha, Venkata Subrahmanyan Govindarajan, Dee Ann Reisinger, Tim Vieira, Keisuke Sakaguchi, Sheng Zhang, Francis Ferraro, Rachel Rudinger, Kyle Rawlins and Benjamin Van Durme. 2020. The Universal Decompositional Semantics Dataset and Decomp Toolkit. Proceedings of the Twelfth Language Resources and Evaluation Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{white2020universal,
  title = {{The Universal Decompositional Semantics Dataset and Decomp Toolkit}},
  author = {Aaron Steven White, Elias Stengel-Eskin, Siddharth Vashishtha, Venkata Subrahmanyan Govindarajan, Dee Ann Reisinger, Tim Vieira, Keisuke Sakaguchi, Sheng Zhang, Francis Ferraro, Rachel Rudinger, Kyle Rawlins and Benjamin {Van Durme}},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference},
  year = {2020},
  doi = {10.13016/m2fhz1-knec},
  url = {https://doi.org/10.13016/m2fhz1-knec}
}

References

25 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Abend, O. and Rappoport, A. (2013). Universal Conceptual Cognitive Annotation (UCCA). In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 228– 238, Sofia, Bulgaria, August. Association for Computational Linguistics.
  2. Abzianidze, L. and Bos, J. (2017). Towards Universal Semantic Tagging. In IWCS 2017 12th International Conference on Computational Semantics Short papers. ↗
  3. Abzianidze, L., Bjerva, J., Evang, K., Haagsma, H., van Noord, R., Ludmann, P., Nguyen, D.-D., and Bos, J. (2017). The Parallel Meaning Bank: Towards a Multilingual Corpus of Translations Annotated with Compositional Meaning Representations. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 242–247, Valencia, Spain, April. Association for Computational Linguistics.
  4. Agresti, A. (2014). Categorical Data Analysis. John Wiley & Sons. find
  5. Banarescu, L., Bonial, C., Cai, S., Georgescu, M., Griffitt, K., Hermjakob, U., Knight, K., Koehn, P., Palmer, M., and Schneider, N. (2013). Abstract meaning representation for sembanking. In Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse, pages 178–186.
  6. Bies, A., Mott, J., Warner, C., and Kulick, S. (2012). English web treebank. Linguistic Data Consortium, Philadelphia, PA. ↗
  7. Bos, J., Basile, V., Evang, K., Venhuizen, N., and Bjerva, J. (2017). The Groningen Meaning Bank. In Nancy Ide et al., editors, Handbook of Linguistic Annotation. Springer, Berlin. ↗
  8. Ciaramita, M. and Johnson, M. (2003). Supersense Tagging of Unknown Nouns in WordNet. In Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing, pages 168–175.
  9. Fellbaum, C. (1998). WordNet. Wiley Online Library. find
  10. Govindarajan, V., Van Durme, B., and White, A. S. (2019). Decomposing Generalization: Models of Generic, Habitual, and Episodic Statements. Transactions of the Association for Computational Linguistics, 7:501–517. ↗
  11. Levin, B. and Rappaport Hovav, M. (2005). Argument Realization. Cambridge University Press, Cambridge. find
  12. Miller, G. A. (1995). WordNet: a lexical database for English. Communications of the ACM, 38(11):39–41. Nivre, J., Agic, Z., Aranzabe, M. J., Asahara, M., Atutxa, A., Ballesteros, M., Bauer, J., Bengoetxea, K., Bhat, R. A., Bosco, C., Bowman, S., Celano, G. G. A., Connor, M., de Marneffe, M.-C., Diaz de Ilarraza, A., Dobrovoljc, K., Dozat, T., Erjavec, T., Farkas, R., Foster, J., Galbraith, D., Ginter, F., Goenaga, I., Gojenola, K., Goldberg, Y., Gonzales, B., Guillaume, B., Haji, J., Haug, D., Ion, R., Irimia, E., Johannsen, A., Kanayama, H., Kanerva, J., Krek, S., Laippala, V., Lenci, A., Ljubei, N., Lynn, T., Manning, C., Mrnduc, C., Mareek, D., Martnez Alonso, H., Maek, J., Matsumoto, Y., Mc-Donald, R., Missil, A., Mititelu, V., Miyao, Y., Montemagni, S., Mori, S., Nurmi, H., Osenova, P., vrelid, L., Pascual, E., Passarotti, M., Perez, C.-A., Petrov, S., Piitulainen, J., Plank, B., Popel, M., Prokopidis, P., Pyysalo, S., Ramasamy, L., Rosa, R., Saleh, S., Schuster, S., Seeker, W., Seraji, M., Silveira, N., Simi, M., Simionescu, R., Simk, K., Simov, K., Smith, A., tpnek, J., Suhr, A., Sznt, Z., Tanaka, T., Tsarfaty, R., Uematsu, S., Uria, L., Varga, V., Vincze, V., abokrtsk, Z., Zeman, D., and Zhu, H. (2015). Universal Dependencies 1.2. http://universaldependencies.github.io/docs/. ↗
  13. Oepen, S., Kuhlmann, M., Miyao, Y., Zeman, D., Flickinger, D., Haji, J., Ivanova, A., and Zhang, Y. (2014). SemEval 2014 Task 8: Broad-Coverage Semantic Dependency Parsing. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 63–72, Dublin, Ireland, August. Association for Computational Linguistics. find
  14. Oepen, S., Kuhlmann, M., Miyao, Y., Zeman, D., Cinkov, S., Flickinger, D., Haji, J., and Ureov, Z. (2015). SemEval 2015 Task 18: Broad-Coverage Semantic Dependency Parsing. In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 915–926, Denver, Colorado, June. Association for Computational Linguistics. find
  15. Palmer, M., Gildea, D., and Kingsbury, P. (2005). The proposition bank: An annotated corpus of semantic roles. Computational Linguistics, 31(1):71–106. ↗
  16. Pustejovsky, J. (1995). The Generative Lexicon. MIT Press, Cambridge, MA. ↗
  17. Reisinger, D., Rudinger, R., Ferraro, F., Harman, C., Rawlins, K., and Van Durme, B. (2015). Semantic Proto-Roles. Transactions of the Association for Computational Linguistics, 3:475–488. ↗
  18. Ross, J. R. (1972). Act. In Donald Davidson et al., editors, Semantics of Natural Language, pages 70–126. Springer. find
  19. Rudinger, R., White, A. S., and Van Durme, B. (2018). Neural Models of Factuality. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 731–744, New Orleans, Louisiana. Association for Computational Linguistics.
  20. Schneider, N., Hwang, J. D., Srikumar, V., Prange, J., Blodgett, A., Moeller, S. R., Stern, A., Bitan, A., and Abend, O. (2018). Comprehensive Supersense Disambiguation of English Prepositions and Possessives. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 185–196, Melbourne, Australia, July. Association for Computational Linguistics. find
  21. Shalev, A., Hwang, J. D., Schneider, N., Srikumar, V., Abend, O., and Rappoport, A. (2019). Preparing SNACS for Subjects and Objects. In Proceedings of the First International Workshop on Designing Meaning Representations, pages 141–147, Florence, Italy, August. Association for Computational Linguistics. find
  22. Vashishtha, S., Van Durme, B., and White, A. S. (2019). Fine-Grained Temporal Relation Extraction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2906–2919, Florence, Italy, July. Association for Computational Linguistics. find
  23. White, A. S., Reisinger, D., Sakaguchi, K., Vieira, T., Zhang, S., Rudinger, R., Rawlins, K., and Van Durme, B. (2016). Universal decompositional semantics on universal dependencies. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1713–1723, Austin, TX. Association for Computational Linguistics.
  24. White, A. S., Rawlins, K., and Van Durme, B. (2017). The Semantic Proto-Role Linking Model. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, volume 2, pages 92–98, Valencia, Spain. Association for Computational Linguistics. find
  25. Zhang, S., Rudinger, R., and Van Durme, B. (2017). An Evaluation of PredPatt and Open IE via Stage 1 Semantic Role Labeling. In IWCS 2017 12th International Conference on Computational Semantics Short papers. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/white-2020-universal-decompositional-semantics-dataset-decomp/