📄 From compositional semantics to Bayesian pragmatics via logical inference

Summary

Montague-style formal semantics assigns precise meanings to sentences but has little to say about how context and background knowledge shape interpretation. Formal pragmatic models such as Rational Speech Act (RSA) predict meaning in context, yet they lack a well-defined compositional semantics. The authors bring the two together.

Their interpreter turns a syntactic analysis into a probability distribution over first-order logic formulae. A set of random parameters θ selects among the possible readings, for example which antecedent a pronoun picks up. Background knowledge is modelled as a distribution over world-states, each a set of FOL formulae. Truth is checked with a depth-bounded tableau prover, which gives expected truth values. A variant of RSA, generalised to arbitrary propositions, then computes a posterior over meanings. All components are written as composable probabilistic programs in continuation-passing (monadic) style, staying within the simply typed λ-calculus.

Applied to pronoun resolution in two scenarios with different priors, the model yields the expected antecedent preferences. Its output stays stable across reasonable parameter ranges. The work shows a principled way for lexical and world knowledge to steer disambiguation without giving up logical semantics.

Key points

• Pipeline: syntax → distribution over FOL formulae (parameterised by θ) → truth values via a FOL tableau prover capped at depth 10 → RSA refinement over meanings. Code is released on GitHub.
• Literal listener: the probability of a reading is the expected chance that the world-state, the candidate proposition and the utterance's interpretation together do not derive a contradiction.
• World knowledge can be continuous or discrete. Examples include a height drawn from a normal distribution (mean 1.8 m, sd 0.05 m) and sets of literals whose polarity is fixed by Bernoulli variables.
• Differences from Lassiter and Goodman (2013):
  • The model infers utterance meanings rather than directly marginalising world-states.
  • The pragmatic speaker re-samples θ instead of keeping it fixed.
  • Unlike Goodman and Lassiter's Church implementation, probability is added without changing the metalanguage's semantics.
• Limitations and future work:
  • Limitations: there are no syntactic binding constraints and no discourse or psycholinguistic factors in the anaphora model.
  • Future work: the authors plan to address vagueness and to evaluate on FraCaS, RTE and the probabilistic suite of Bernardy et al. (2019a).

Interesting ideas

• I1. Treat each ambiguity as an explicit random parameter of the interpretation function, so the semantics outputs a weighted set of logical forms rather than committing early.
• I2. Score a candidate reading by whether it stays consistent with sampled background world-states, i.e. a contradiction check rather than a similarity score.
• I3. Bound the theorem prover's depth, both to sidestep undecidability and as a model of finite reasoning by agents.
• I4. Represent uncertain world knowledge as sets of literals whose truth is drawn from Bernoulli variables, plus continuous distributions for measures.
• I5. Check that model predictions remain stable when its free parameters (rationality exponent, costs, priors) are varied within plausible ranges.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (frames): Pronoun antecedents and attachment choices could become named parameters in frame building, keeping alternative frames alive until world knowledge decides between them.
• I2 (world): Each candidate frame could be added to the text's world and rejected or down-weighted when it clashes with known states or holdings, which gives an explainable reason for the choice.
• I3 (answer): Readers answering test questions could run inference under a fixed depth budget, guaranteeing termination and short, inspectable proof traces.
• I4 (world): Uncertain knowledge-graph facts and measures (heights, prices) could carry such priors, so expected truth values replace hard yes/no lookups.
• I5 (learn): Evaluation could include parameter sweeps over tunable pragmatic weights, flagging answers that flip under small changes as fragile.