Elamur Research⌕
☰
home / papers / 2021 / Calibrate your listeners! Robust communication-based training for pragmatic speakers

Calibrate your listeners! Robust communication-based training for pragmatic speakers

Paper · 2021 · Rose Wang, Julia White, Jesse Mu et al.

Summary

The paper studies pragmatic speakers trained by communication: a speaker learns to describe a target image so that a learned neural listener can pick it out from distractors. This kind of training tends to cause semantic drift, where the speaker invents a private code that pleases its training partner but no longer resembles English. The authors use ShapeWorld reference games with one target and two distractors. They show that drift appears once the vocabulary grows from 15 ShapeWorld tokens to the 51k GPT-2 BPE vocabulary. They trace the cause to poor listener calibration: one neural listener gives confident answers even for strange, out-of-domain strings, and the speaker learns to exploit this. Their fix is to train the speaker against a population of listeners. A randomly initialised ensemble, trained on different splits, is better calibrated than an MC-dropout population. With the ensemble, speakers generalise to held-out listeners and use more domain words. The broader lesson is that a model used as a reward must know when it does not understand.

Key points

  • Setup:
    • 15,000 random ShapeWorld games, each with 1 target and 2 distractors.
    • The speaker is a GRU conditioned on image encodings.
    • The listener scores each image by the dot product of image and utterance encodings.
    • Listeners and speakers are trained on disjoint data splits, and validation listeners are kept out of speaker training.
  • Small vocabulary: the speaker reaches 99/90% with the training listener and 80/80% with a held-out listener. Token overlap with ShapeWorld captions is 100%.
  • GPT-2 vocabulary: the speaker reaches 98/94% with the training listener but only 37/36% with a held-out listener, close to the 33% chance level. Token overlap falls to about 1%, and outputs become strings like "Macedichickarios".
  • Calibration (entropy against domain overlap):
    • A single listener is overconfident on low-overlap utterances.
    • MC-dropout with p = 0.1 does not fix this.
    • An ensemble, whose members each reach at least 92% accuracy, comes closer to the ideal pattern.
  • Ensemble results:
    • As the ensemble grows (n ∈ {1, 10, 20, 30}, 10 seeds), the gap between training and held-out listeners closes.
    • At n = 30, token overlap reaches 48.4 ± 12.3%.
    • GloVe distance to ground-truth captions is 13.03 for the miscalibrated speaker, 10.37 for the calibrated one and 9.2 for the vocabulary-limited one (summed embeddings).

Interesting ideas

  • I1. Check whether a scorer is calibrated by plotting its entropy against how far inputs lie from its training domain, using in-domain, medium and garbage inputs; a usable judge should become unsure as inputs drift.
  • I2. An ensemble of independently trained models, built from different splits and seeds, catches out-of-domain inputs through disagreement, while cheap dropout sampling of a single model mostly fails to.
  • I3. Always evaluate an optimised producer against a held-out consumer it never trained with; success with the training partner alone can hide a private code.
  • I4. Measure drift with a simple domain-vocabulary overlap rate alongside task accuracy, so that degenerate but "successful" outputs show up.
  • I5. When comparing utterance meaning with embedding sums, also compare only the first word, because repeated tokens distort a summed vector.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): Apply this test to each witness parser and to the LLM teacher tiers, feeding them deliberately corrupted sentences as negative controls, before trusting their agreement as silver data.
  • I2 (tree): Train several perceptron or gparse parsers on different treebank splits, and use their disagreement to flag sentences where no single parser's confidence should be trusted.
  • I3 (back): Score generated text by whether an independent reader (a separate parse → frames pass or a different model) recovers the same meaning graph, not only the reader used during tuning.
  • I4 (back): During compressor or generation self-play, track the share of output lemmas that come from the source world's vocabulary, as an alarm for drift into gibberish.
  • I5 (learn): In evaluation scripts that compare generated answers with references, add a repetition-robust variant of the metric so that looping outputs are not scored unfairly well or badly.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Stanford UniversityStanford · United States310 works · 284,480 citations

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

🧭 Topics 1Research subjects and languages the work is about.English · 1,373 works

Cite

Cite as Rose Wang, Julia White, Jesse Mu and Noah Goodman. 2021. Calibrate your listeners! Robust communication-based training for pragmatic speakers. Findings of the Association for Computational Linguistics: EMNLP 2021.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2021.findings-emnlp.83 · arXiv 2110.05422

@inproceedings{wang2021calibrate,
  title = {{Calibrate your listeners! Robust communication-based training for pragmatic speakers}},
  author = {Rose Wang, Julia White, Jesse Mu and Noah Goodman},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2021},
  year = {2021},
  doi = {10.18653/v1/2021.findings-emnlp.83},
  eprint = {2110.05422},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2021.findings-emnlp.83}
}

References

28 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Angeliki Lazaridou, Alexander Peysakhovich, and Marco Baroni. 2017. Multi-agent cooperation and the emergence of (natural) language. In International Conference on Learning Representations (ICLR). ↗
  2. Jacob Andreas and Dan Klein. 2016. Reasoning about pragmatics with neural listeners and speakers. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1173–1182.
  3. William H Beluch, Tim Genewein, Andreas Nürn-berger, and Jan M Köhler. 2018. The power of ensembles for active learning in image classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 9368– 9377. find
  4. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. 2003. A neural probabilistic language model. The journal of machine learning research, 3:1137–1155.
  5. Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using rnn encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1724– 1734. ↗
  6. Angeliki Lazaridou, Anna Potapenko, and Oliver Tieleman. 2020. Multi-agent communication meets natural language: Synergies between functional and structural language learning. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL). find
  7. Jason Lee, Kyunghyun Cho, and Douwe Kiela. 2019. Countering language drift via visual grounding. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4385– 4395. find
  8. D. Lewis. 1969. Convention: A Philosophical Study. Harvard University Press, Cambridge, MA. find
  9. Fushan Li and Michael Bowling. 2019. Ease-of-teaching and language structure from emergent communication. Advances in Neural Information Processing Systems, 32:15851–15861. ↗
  10. Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pages 1050–1059. PMLR. ↗
  11. Will Monroe, Robert XD Hawkins, Noah D Goodman, and Christopher Potts. 2017. Colors in context: A pragmatic neural model for grounded language understanding. Transactions of the Association for Computational Linguistics, 5:325–338. ↗
  12. Noah D Goodman and Michael C Frank. 2016. Pragmatic language interpretation as probabilistic inference. Trends in Cognitive Sciences, 20(11):818– 829. ↗
  13. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543. ↗
  14. Laura Graesser, Kyunghyun Cho, and Douwe Kiela. 2020. Emergent linguistic phenomena in multi-agent communication games. In 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, pages 3700–3710. Association for Computational Linguistics. find
  15. H. P. Grice. 1975. Logic and conversation. In P. Cole and J. L. Morgan, editors, Syntax and Semantics: Vol. 3: Speech Acts. Academic Press, New York. ↗
  16. Robert XD Hawkins, Mike Frank, and Noah D Goodman. 2017. Convention-formation in iterated reference games. In CogSci. ↗
  17. E. Jang, S. Gu, and B. Poole. 2017. Categorical reparameterization with gumbel-softmax. In International Conference on Learning Representations (ICLR). ↗
  18. A. Kuhnle and A. Copestake. 2017. ShapeWorld - a new test methodology for multimodal language understanding. arXiv preprint. ↗
  19. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8). find
  20. Limor Raviv, Antje Meyer, and Shiri Lev-Ari. 2019. Larger communities create more systematic languages. Proceedings of the Royal Society B, 286(1907):20191262. ↗
  21. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715– 1725, Berlin, Germany. Association for Computational Linguistics. ↗
  22. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. 2020. Learning to summarize with human feedback. In Advances in Neural Information Processing Systems, volume 33, pages 3008–3021. Curran Associates, Inc. find
  23. Olivier Tieleman, Angeliki Lazaridou, Shibl Mourad, Charles Blundell, and Doina Precup. 2019. Shaping representations through communication: community size effect in artificial learning systems. arXiv. ↗
  24. Ramakrishna Vedantam, Samy Bengio, Kevin Murphy, Devi Parikh, and Gal Chechik. 2017. Context-aware captions from context-agnostic supervision. In Computer Vision and Pattern Recognition (CVPR), pages 251–260. find
  25. Kyle Wagner, James A Reggia, Juan Uriagereka, and Gerald S Wilkinson. 2003. Progress in the simulation of emergent communication and language. Adaptive Behavior, 11(1):37–69. ↗
  26. Julia White, Jesse Mu, and Noah D. Goodman. 2020. Learning to refer informatively by amortizing pragmatic reasoning. In CogSci. ↗
  27. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtow-icz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. Huggingface’s transformers: State-of-the-art natural language processing. ↗
  28. Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. 2020. Fine-tuning language models from human preferences. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2021-calibrate-your-listeners-robust-communication/