Elamur Research⌕
☰
home / papers / 2015 / A Frobenius Model of Information Structure in Categorical Compositional Distributional Semantics

A Frobenius Model of Information Structure in Categorical Compositional Distributional Semantics

Paper · 2015 · Dimitri Kartsaklis and Mehrnoosh Sadrzadeh

Summary

The categorical compositional distributional framework of Coecke et al. (2010) builds sentence vectors by letting a pregroup derivation drive tensor contraction over word vectors and tensors. Until now it has captured only syntax. It has ignored information structure, meaning the split of an utterance into a theme (shared, known content) and a rheme (content that is new to the listener). English often marks this split with intonation, and the split need not follow constituent boundaries. Kartsaklis and Sadrzadeh extend the framework by treating intonational boundaries as words with new pregroup types built from θ (theme) and ρ (rheme). These boundaries are interpreted in FVect through the multiplication of the Frobenius algebra that a fixed basis induces. The sentence meaning then becomes an element-wise product of the theme and rheme representations, so both parts contribute equally. The model thus has two composition operators: tensor contraction for syntax and Frobenius multiplication for information structure. The paper is purely theoretical. It gives truth-theoretic and distributional readings and works through many examples, and leaves empirical testing, ideally on question answering, to future work.

Key points

  • The approach starts from Lambek's pregroup grammar and a strongly monoidal passage F from pregroups to FVect. In this passage:
    • nouns are vectors in N;
    • intransitive verbs live in N ⊗ S;
    • transitive verbs are order-3 tensors in N ⊗ S ⊗ N.
  • Intonational boundaries receive new types, for example θʳ·s·ρˡ, and are interpreted by the Frobenius multiplication µ. Under a fixed basis, µ works as element-wise multiplication.
  • Postulate 4.1 states that, under information structure, intonational boundaries determine the intended syntactic structure. This lets themes that are not constituents still compose, as in "[Mary wrote a book about]T [ART]R".
  • The paper covers several configurations:
    • Multiple rhemes ("JOHN likes MARY"): the result is a matrix rather than a vector, so sentences with different numbers of rhemes cannot be compared.
    • Verb as rheme: collapses to the same vector as the multiple-rheme case because the Frobenius operations are commutative. Non-commutative versions (Piedeleu et al., 2015) are suggested as a fix.
    • Rhemes in mid-sentence: handled through the spider equality.
    • Nested rhemes.
  • There is no experimental evaluation. A question-answering evaluation and embedding the boundary semantics into the functor itself are left as future work.

Interesting ideas

  • I1. The paper keeps two separate composition operators: one follows grammar and the other follows the given-versus-new split. Information structure becomes its own explicit layer instead of being folded into syntax.
  • I2. The paper uses the question to decide what counts as rheme ("Who likes whom?" picks out JOHN and MARY). The question context then fixes which part of the answer is new.
  • I3. The paper lets an information-structure boundary override the standard parse when the theme is not a constituent. This acknowledges that a pragmatic segmentation can legitimately cut across a syntactic one.
  • I4. In this model the shape of the result depends on how many rhemes a sentence has, which makes the incomparability of different focus structures explicit rather than hidden.
  • I5. The paper notes that commutative operators erase the distinction between "JOHN likes MARY" and "John LIKES Mary", which is a good reminder to test whether a combination operator destroys order or role information.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): prag could annotate each frame with a theme/rheme layer kept apart from the UD-derived argument structure, making focus explicit and explainable.
  • I2 (answer): QA readers could use the question's wh-slots to mark the expected rheme span in candidate answers and check that the answer fills exactly that slot.
  • I3 (frames): allow a theme span over non-constituent UD subtrees (e.g. "Mary wrote a book about"), linked to the tree by node sets rather than by forcing a single subtree.
  • I4 (world): when comparing or merging statements in the world model, first check that their focus structures match, and treat mismatches as different question types rather than near-duplicates.
  • I5 (learn): add negative-control test pairs that differ only in focus placement, and check that representations or answers distinguish them.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of OxfordOxford · United Kingdom52 works · 154,829 citations🎓Queen Mary University of LondonLondon · United Kingdom39 works · 3,307 citations

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksComputational Theory and Mathematics · 116 works

Cite

Cite as Dimitri Kartsaklis and Mehrnoosh Sadrzadeh. 2015. A Frobenius Model of Information Structure in Categorical Compositional Distributional Semantics. Proceedings of the 14th Meeting on the Mathematics of Language (MoL 2015).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/w15-2306 · arXiv 1505.06294

@inproceedings{kartsaklis2015frobenius,
  title = {{A Frobenius Model of Information Structure in Categorical Compositional Distributional Semantics}},
  author = {Dimitri Kartsaklis and Mehrnoosh Sadrzadeh},
  booktitle = {Proceedings of the 14th Meeting on the Mathematics of Language (MoL 2015)},
  year = {2015},
  doi = {10.3115/v1/w15-2306},
  eprint = {1505.06294},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.3115/v1/w15-2306}
}

References

27 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Wallace L. Chafe. 1976. Givenness, contrastiveness, definiteness, subjects, topics and point of view. In Charles N. Li, editor, Subject and Topic. Academic Press, New York. find
  2. B. Coecke, M. Sadrzadeh, and S. Clark. 2010. Mathematical Foundations for a Compositional Distributional Model of Meaning. Lambek Festschrift. Linguistic Analysis, 36:345–384. ↗
  3. Caroline Féry and Manfred Krifka. 2008. Information Structure: Notional distinctions, ways of expression. In Piet van Sterkenburg, editor, Unity and diversity of languages, pages 123–135. John Benjamins Publishing. ↗
  4. Ines Fiedler, Katharina Hartmann, Brigitte Reineke, Anne Schwarz, and Malte Zimmermann. 2006. Subject focus in West African languages. In International Conference of Information Structure, Potsdam. ↗
  5. Z. Harris. 1968. Mathematical Structures of Language. Wiley. find
  6. Dimitri Kartsaklis, Mehrnoosh Sadrzadeh, and Stephen Pulman. 2012. A unified sentence space for categorical distributional-compositional semantics: Theory and experiments. In Proceedings of 24th International Conference on Computational Linguistics (COLING 2012): Posters, pages 549–558, Mumbai, India, December. find
  7. The COLING 2012 Organizing Committee. find
  8. Dimitri Kartsaklis, Mehrnoosh Sadrzadeh, Stephen Pulman, and Bob Coecke. 2014. Reasoning about meaning in natural language with compact closed categories and Frobenius algebras. arXiv preprint arXiv:1401.5980. ↗
  9. Dimitri Kartsaklis. 2015. Compositional Distributional Semantics with Compact Closed Categories and Frobenius Algebras. Ph.D. thesis, University of Oxford. ↗
  10. G Maxwell Kelly. 1972. Many-variable functorial calculus (I). In G.M. Kelly, M. Laplaza, G. Lewis, and S. MacLane, editors, Coherence in categories, pages 66–105. Springer. ↗
  11. J. Lambek. 2008. From Word to Sentence. Polimetrica, Milan. find
  12. T. Landauer and S. Dumais. 1997. A Solution to Plato’s Problem: The Latent Semantic Analysis Theory of Acquision, Induction, and Representation of Knowledge. Psychological Review. find
  13. Mike Lewis and Mark Steedman. 2013. Combined distributional and logical semantics. Transactions of the Association for Computational Linguistics, 1:179– 192. ↗
  14. Saunders Mac Lane. 1971. Categories for the Working Mathematician. Number 5 in Graduate Texts in Mathematics. Springer-Verlag. ↗
  15. C.D. Manning, P. Raghavan, and H. Schütze. 2008. Introduction to Information Retrieval. Cambridge University Press. ↗
  16. J. Mitchell and M. Lapata. 2008. Vector-based Models of Semantic Composition. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics, pages 236–244. find
  17. Richard Montague. 1970. English as a formal language. In Linguaggi nella Società e nella Tecnica, pages 189– 224. Edizioni di Comunità, Milan. find
  18. Robin Piedeleu, Dimitri Kartsaklis, Bob Coecke, and Mehrnoosh Sadrzadeh. 2015. Open System Categorical Quantum Semantics in Natural Language Processing. In Proceedings of the 6th Conference on Algebra and Coalgebra in Computer Science (CALCO), Nijmegen, Netherlands, June. ↗
  19. Janet Pierrehumbert and Julia Hirschberg. 1990. The meaning of intonational contours in the interpretation of discourse. In Philip Cohen, Jerry Morgan, and Martha Pollack, editors, Intentions in communication, pages 271–312. MIT Press, Cambridge MA. ↗
  20. Mats Rooth. 1992. A theory of focus interpretation. Natural language semantics, 1(1):75–116. ↗
  21. M. Sadrzadeh, S. Clark, and B. Coecke. 2013. The Frobenius anatomy of word meanings I: subject and object relative pronouns. Journal of Logic and Computation, Advance Access, October. ↗
  22. M. Sadrzadeh, S. Clark, and B. Coecke. 2014. The Frobenius anatomy of word meanings II: Possessive relative pronouns. Journal of Logic and Computation, June. ↗
  23. H. Schütze. 1998. Automatic Word Sense Discrimination. Computational Linguistics, 24:97–123.
  24. Peter Selinger. 2011. A survey of graphical languages for monoidal categories. In Bob Coecke, editor, New structures for physics, pages 289–355. Springer. ↗
  25. Mark Steedman. 2000. Information structure and the syntax-phonology interface. Linguistic inquiry, 31(4):649–689. ↗
  26. Mark Steedman. 2001. The Syntactic Process. MIT Press. ↗
  27. Peter D Turney and Patrick Pantel. 2010. From frequency to meaning: Vector space models of semantics. Journal of artificial intelligence research, 37(1):141– 188. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kartsaklis-2015-frobenius-model-information-structure-categorical/