Elamur Research⌕
☰
home / papers / 2024 / Regularized Conventions: Equilibrium Computation as a Model of Pragmatic Reasoning

Regularized Conventions: Equilibrium Computation as a Model of Pragmatic Reasoning

Paper · 2024 · Athul Jacob, Gabriele Farina and Jacob Andreas

Summary

Computational pragmatics usually explains implicature with recursive reasoning models such as RSA or iterated best response. Their predictions depend heavily on the starting policy and on how many reasoning steps are run. Plain signaling games have the opposite problem: they allow many optimal conventions, including odd ones that ignore what words ordinarily mean. The authors propose ReCo, which treats speaking and interpreting as a search for a regularized equilibrium. Each agent maximizes expected communicative success minus a KL penalty for straying from a "default semantics" policy, with the penalty weighted by λ_S or λ_L. The equilibrium is computed with piKL-Hedge, and policies are stored as row-stochastic matrices. ReCo is tested on three kinds of data: classic implicature exemplars, graded human reference judgments from Frank (2016), and the Colors in Context dataset with transformer listener and speaker base models. Its fit to human behavior matches, and sometimes modestly exceeds, RSA and RD-RSA. The approach matters because it replaces procedural recursion with an explicit objective that has formal guarantees. This links modern game-solving tools to learned pragmatic models.

Key points

  • Each agent's utility is its expected reward minus λ_i·D_KL(π_i ∥ τ_i). A large λ keeps the policy near the default semantics τ, and a λ near zero leaves it almost unconstrained.
  • Equilibria are found with piKL-Hedge (Jacob et al., 2022), previously used for Diplomacy and for language-model decoding. Its update structure closely resembles recursive RSA-style reasoning.
  • Colors in Context (Monroe et al., 2017) has 46,994 utterances, split 80/10/10. Each context gets 16 candidate utterances: 5 sampled per each of the 3 targets, plus the human utterance. The metric is top-1 accuracy of recovering the intended referent.
  • On Colors in Context, every model except BR performs well, including the literal listener, and ReCo equals or slightly improves on these results. On the Frank (2016) graded judgments, the fit is measured with Pearson's ρ across sweeps of λ_L and λ_S, against RSA and RD-RSA at varying α and recursion depth.
  • Stated limitation: the method assumes finite sets of utterances and meanings, so open-ended generation would need further work.

Interesting ideas

  • I1. Instead of tuning a recursion depth, pragmatic interpretation is framed as maximizing communicative success under a KL leash to default readings, with λ as one interpretable tradeoff knob.
  • I2. Speaker and listener policies are encoded as row-stochastic matrices over a small, explicit set of meanings and utterances, which makes the whole computation vectorized and easy to inspect.
  • I3. An open-ended problem becomes finite by sampling candidate utterances from a generative speaker for every possible target, plus the observed one, and then reasoning only over that pool.
  • I4. Baselines are given full sweeps over their own hyperparameters (α, depth) and compared with the regularized model at its best setting, and a literal listener serves as a sanity floor.
  • I5. The authors suggest modeling conventions that form over several rounds by setting the next round's default semantics to the previous round's equilibrium.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): prag could choose implicature readings by trading off success against distance from the literal frame reading, logging λ and the KL cost as part of the explanation.
  • I2 (frames): coreference could use candidate antecedents as "meanings" and mentions as "utterances", with a listener matrix whose rows can be printed directly as evidence.
  • I3 (back): generation could sample referring expressions for each entity and keep the one that Mova's own reader resolves to the intended entity.
  • I4 (learn): pragmatic modules should always be reported against a literal, no-pragmatics control tuned just as fairly, so gains are not artifacts of uneven tuning.
  • I5 (world): within one text, once a descriptive label has been resolved to an entity, that mapping could become the default for later mentions in the text world.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,426 works

Cite

Cite as Athul Jacob, Gabriele Farina and Jacob Andreas. 2024. Regularized Conventions: Equilibrium Computation as a Model of Pragmatic Reasoning. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.naacl-long.163 · arXiv 2311.09712

@inproceedings{jacob2024regularized,
  title = {{Regularized Conventions: Equilibrium Computation as a Model of Pragmatic Reasoning}},
  author = {Athul Jacob, Gabriele Farina and Jacob Andreas},
  booktitle = {Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)},
  year = {2024},
  doi = {10.18653/v1/2024.naacl-long.163},
  eprint = {2311.09712},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2024.naacl-long.163}
}

References

31 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. James Hannan. 1957. Approximation to Bayes risk in repeated play. Contributions to the Theory of Games, 3:97–139. find
  2. Anton Bakhtin, David J Wu, Adam Lerer, Jonathan Gray, Athul Paul Jacob, Gabriele Farina, Alexander H Miller, and Noam Brown. 2022. Mastering the game of no-press Diplomacy via human-regularized reinforcement learning and planning. In Proceedings of the International Conference on Learning Representations. ↗
  3. Leon Bergen, Roger Levy, and Noah Goodman. 2016. Pragmatic reasoning through semantic inference. Semantics and Pragmatics, 9. ↗
  4. Herbert H Clark and Deanna Wilkes-Gibbs. 1986. Referring as a collaborative process. Cognition, 22(1):1– 39. ↗
  5. Judith Degen. 2023. The rational speech act framework. Annual Review of Linguistics, 9:519–540. ↗
  6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4171–4186.
  7. Meta Fundamental AI Research Diplomacy Team FAIR, Anton Bakhtin, Noam Brown, Emily Dinan, Gabriele Farina, Colin Flaherty, Daniel Fried, Andrew Goff, Jonathan Gray, Hengyuan Hu, et al. 2022. Human-level play in the game of diplomacy by combining language models with strategic reasoning. Science, 378(6624):1067–1074. find
  8. Michael C Frank. 2016. Rational speech act models of pragmatic reasoning in reference games. ↗
  9. Robert Hawkins, Minae Kwon, Dorsa Sadigh, and Noah Goodman. 2020. Continual adaptation for efficient machine communication. In Proceedings of the 24th Conference on Computational Natural Language Learning, pages 408–419.
  10. Robert D Hawkins, Michael Franke, Michael C Frank, Adele E Goldberg, Kenny Smith, Thomas L Griffiths, and Noah D Goodman. 2023. From partners to populations: A hierarchical bayesian account of coordination and convention. Psychological Review, 130(4):977. ↗
  11. Robert XD Hawkins, Mike Frank, and Noah D Goodman. 2017. Convention-formation in iterated reference games. In Proceedings of the Annual Meeting of the Cognitive Science Society. ↗
  12. Yan Huang. 1991. A neo-gricean pragmatic theory of anaphora1. Journal of linguistics, 27(2):301–335. find
  13. Athul Paul Jacob, Abhishek Gupta, and Jacob Andreas. 2023a. Modeling boundedly rational agents with latent inference budgets. arXiv preprint arXiv:2312.04030. ↗
  14. Athul Paul Jacob, Yikang Shen, Gabriele Farina, and Jacob Andreas. 2023b. The consensus game: Language model generation via equilibrium search. arXiv preprint arXiv:2310.09139. ↗
  15. Athul Paul Jacob, David J Wu, Gabriele Farina, Adam Lerer, Hengyuan Hu, Anton Bakhtin, Jacob Andreas, and Noam Brown. 2022. Modeling strong and human-like gameplay with KL-regularized search. In Proceedings of the International Conference on Machine Learning. ↗
  16. Gerhard Jäger. 2007. Game dynamics connects semantics and pragmatics. In Game Theory and Linguistic Meaning, pages 103–117. find
  17. Gerhard Jäger. 2008a. Applications of game theory in linguistics. Language and Linguistics compass, 2(3):406–421. find
  18. Gerhard Jäger. 2008b. Evolutionary stability conditions for signaling games with costly signals. Journal of theoretical biology, 253(1):131–141. ↗
  19. Noga Zaslavsky, Jennifer Hu, and Roger Levy. 2021a. A rate–distortion view of human pragmatic reasoning. In Proceedings of the Society for Computation in Linguistics. ↗
  20. Noga Zaslavsky, Jennifer Hu, and Roger P. Levy. 2021b. A Rate–Distortion view of human pragmatic reasoning? In Proceedings of the Society for Computation in Linguistics 2021, pages 347–348, Online. Association for Computational Linguistics. ↗
  21. Gerhard Jäger. 2012. Game theory in semantics and pragmatics. Semantics: An International Handbook of Natural Language Meaning, 3:2487–2516. find
  22. Stephen C Levinson. 2000. Presumptive meanings: The theory of generalized conversational implicature. ↗
  23. David K Lewis. 1971. Convention: A philosophical study. Philosophy and Rhetoric, 4(2). find
  24. Nick Littlestone and Manfred K Warmuth. 1994. The weighted majority algorithm. Information and Computation, 108(2):212–261. ↗
  25. Bill McDowell and Noah Goodman. 2019. Learning from omission. In Proceedings of the Annual Meeting of the Association for Computational Linguistics. find
  26. Will Monroe, Robert XD Hawkins, Noah D Goodman, and Christopher Potts. 2017. Colors in context: A pragmatic neural model for grounded language understanding. Transactions of the Association for Computational Linguistics, 5:325–338. ↗
  27. Prashant Parikh. 2000. Communication, meaning, and interpretation. Linguistics and Philosophy, pages 185–212. ↗
  28. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. PyTorch: An imperative style, high-performance deep learning library. Advances in Neural Information Processing Systems, 32. ↗
  29. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485–5551. ↗
  30. Peter E Trapa and Martin A Nowak. 2000. Nash equilibria for an evolutionary language game. Journal of mathematical biology, 41(2):172–188. ↗
  31. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/jacob-2024-regularized-conventions-equilibrium-computation-as/