Predicting Pragmatic Reasoning in Language Games

Written for the Research library (claude-opus-5-5) from the paper text, in our own words: the license "© AAAS (Science); author-posted manuscript on the author's page, no open licence — readonly" does not allow redistributing the text itself. Original — web.stanford.edu~ngoodman/papers/FrankGoodman-Science2012.pdf

Summary

People often understand what a speaker means even when the words alone are ambiguous. Theories of pragmatics explain this informally, but few make precise, testable predictions. Frank and Goodman address this with very simple reference games. Participants see a small set of objects that vary along two dimensions and bet on which object a single word refers to. The model treats the listener as a Bayesian reasoner that combines two parts. The first is a prior over objects, set to their contextual salience, which a separate group of participants measured directly. The second is a likelihood from a speaker who prefers informative words. Informativeness is defined through surprisal, so a word is chosen with probability inversely proportional to how many objects it could describe. Separate groups of participants gave speaker choices, salience ratings and listener interpretations. The combined model has no free parameters and matched listener bets almost perfectly. The paper is an early foundation of the Rational Speech Acts (RSA) framework. It shows that information-theoretic tools can make pragmatic inference measurable and predictable.

Key points

• Three participant groups in a between-subjects design: speaker (which word describes an object), salience (which object an unknown word refers to, used as the prior) and listener (which object a known word such as "blue" refers to).
• The speaker likelihood comes from a rational-actor model with surprisal-based utility: P(w | r, C) ∝ |w|⁻¹, where |w| is the number of objects in context that the word fits.
• The model's speaker predictions correlated with human speaker bets at r = .98. Listener predictions correlated with listener bets at r = .99, and still at r = .87 when predictions of 0 and 100 were left out.
• Salience judgments on their own did not predict listener judgments (r = .19, p = .40). The good fit appears only when salience is combined with speaker informativeness.
• The model unifies earlier work on disambiguation models, game-theoretic signaling and referring-expression generation. It appeared as a short report in Science 336(6084), 2012.