Elamur Research⌕
☰
home / papers / 2023 / Computational Language Acquisition with Theory of Mind

Computational Language Acquisition with Theory of Mind

Summary

Children learn language through social interaction, partly by reasoning about what other people think, an ability called Theory of Mind (ToM). Standard language models lack this mechanism. This paper tests whether ToM and harder communicative environments help a simulated learner acquire language. The setup is an image referential game. A speaker, modelled as a ResNet encoder with an LSTM decoder, must describe a target image in English so that a pretrained listener, acting as a caregiver, can pick it out from distractors. The speaker is trained with PPO on game rewards and also learns from captions the listener sometimes supplies. For ToM, the speaker contains its own internal listener that is trained to imitate the real one. It samples several candidate utterances and reranks them by how likely this internal listener is to choose the target. When the ToM component is weighted heavily, accuracy and fluency improve. Training with more similar distractors produces longer, more precise utterances, though accuracy does not always rise. Speaker output still falls short of human captions. Overall, the work links developmental psychology to grounded language learning.

Key points

  • Reward scheme: The speaker gets +1 for a correct listener choice and −1 for a wrong one. A smaller penalty (w_noop) applies when the listener declines to choose.
  • Listener feedback: Two confidence thresholds (θ1, θ2) control the listener. Below θ1 it abstains. Between θ1 and θ2 it picks an image and gives the ground-truth caption. Above θ2 it picks without feedback.
  • Speaker limits: Vocabulary is capped at 200 tokens and utterances at 20 tokens, to resemble an early learner.
  • Distractor difficulty: Harder distractors were sampled by image or caption similarity, using CLIP, RoBERTa and TF-IDF variants, plus hybrids. Appendix Table 3 compares them:
DistractorsAccuracyFluencyAverage length
Base0.811.508.97
Caption RoBERTa TFIDF0.86 (best)——
Hybrid TFIDF—2.19—
Human captions—2.5210.79
  • Availability: Code and data are released at github.com/neulab/ToM-Language-Acquisition.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Natural language processing · AI & machine learning

Topics Psycholinguistics & language acquisition

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States278 works · 138,810 citations

Also on the byline: Harvey Mudd College.

Cite

Cite as Andy Liu, Hao Zhu, Emmy Liu, Yonatan Bisk and Graham Neubig. 2023. Computational Language Acquisition with Theory of Mind. arXiv:2303.01502.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2303.01502 · arXiv 2303.01502

@misc{liu2023computational,
  title = {{Computational Language Acquisition with Theory of Mind}},
  author = {Andy Liu, Hao Zhu, Emmy Liu, Yonatan Bisk and Graham Neubig},
  year = {2023},
  doi = {10.48550/arxiv.2303.01502},
  eprint = {2303.01502},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2303.01502}
}

References

34 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jacob Andreas and Dan Klein. Reasoning about pragmatics with neural listeners and speakers. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 1: 1173–1182, 2016. our card
  2. Nolan Bard, Jakob N Foerster, Sarath Chandar, Neil Burch, Marc Lanctot, H Francis Song, Emilio Parisotto, Vincent Dumoulin, Subhodeep Moitra, Edward Hughes, et al. The hanabi challenge: A new frontier for ai research. Artificial Intelligence, 280:103216, 2020. find
  3. Elika Bergelson and Richard Aslin. Semantic specificity in one-year-olds’ word comprehension. Language Learning and Development, 13:1–21, 06 2017a. doi: 10.1080/15475441.2017. 1324308. — [doi:10.1080/15475441.2017](https://doi.org/10.1080/15475441.2017) find
  4. Elika Bergelson and Richard N. Aslin. Nature and origins of the lexicon in 6-mo-olds. Proceedings of the National Academy of Sciences, 114(49):12916–12921, 2017b. doi: 10. 1073/pnas.1712966114. URL https://www.pnas.org/doi/abs/10.1073/pnas. 1712966114. — [doi:10.1073/pnas](https://doi.org/10.1073/pnas) ↗
  5. Steven Bird, Ewan Klein, and Edward Loper. Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit. O’Reilly, Beijing, 2009. ISBN 978-0-596-51649-9. doi: http://my.safaribooksonline.com/9780596516499. URL http://www.nltk.org/book. ↗
  6. Kris Cao, Angeliki Lazaridou, Marc Lanctot, Joel Z Leibo, Karl Tuyls, and Stephen Clark. Emergent communication through negotiation. arXiv preprint arXiv:1804.03980, 2018. — [arXiv:1804.03980](https://arxiv.org/abs/1804.03980) find
  7. Rahma Chaabouni, Florian Strub, Florent Altché, Eugene Tarassov, Corentin Tallec, Elnaz Davoodi, Kory Wallace Mathewson, Olivier Tieleman, Angeliki Lazaridou, and Bilal Piot. Emergent communication at scale. In International Conference on Learning Representations, 2022. URL https://openreview.net/f… ↗
  8. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, 2016. find
  9. Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. spacy: Industrial-strength natural language processing in python. 2020. doi: 10.5281/zenodo.1212303. find
  10. Katharina Kann, Sascha Rothe, and Katja Filippova. Sentence-level fluency evaluation: References help, but can be spared! In Proceedings of the 22nd Conference on Computational Natural Language Learning, pp. 313–323, 2018. find
  11. Angeliki Lazaridou and Marco Baroni. Emergent multi-agent communication in the deep learning era. CoRR, abs/2006.02419, 2020. URL https://arxiv.org/abs/2006.02419. — [arXiv:2006.02419](https://arxiv.org/abs/2006.02419) ↗
  12. Angeliki Lazaridou, Alexander Peysakhovich, and Marco Baroni. Multi-agent cooperation and the emergence of (natural) language. arXiv preprint arXiv:1612.07182, 2016. — [arXiv:1612.07182](https://arxiv.org/abs/1612.07182) find
  13. Angeliki Lazaridou, Anna Potapenko, and Olivier Tieleman. Multi-agent communication meets natural language: Synergies between functional and structural language learning. CoRR, abs/2005.07064, 2020. URL https://arxiv.org/abs/2005.07064. ↗
  14. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740–755. Springer, 2014. our card
  15. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Ro{bert}a: A robustly optimized {bert} pre-training approach, 2020. URL https://openreview.net/forum?id=SyxS0T4tvS. ↗
  16. Ryan Lowe, Abhinav Gupta, Jakob Foerster, Douwe Kiela, and Joelle Pineau. On the interaction between supervision and self-play in emergent communication. In International Conference on Learning Representations, 2019. find
  17. Will Monroe, Robert X.D. Hawkins, Noah D. Goodman, and Christopher Potts. Colors in context: A pragmatic neural model for grounded language understanding. Transactions of the Association for Computational Linguistics, 5:325–338, 2017. doi: 10.1162/tacl a 00064. URL https://aclanthology.org/Q17-1023… ↗ our card
  18. Jesse Mu and Noah Goodman. Emergent communication of generalizations. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 17994–18007. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc… ↗
  19. Aida Nematzadeh, Kaylee Burns, Erin Grant, Alison Gopnik, and Tom Griffiths. Evaluating theory of mind in question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2392–2400, Brussels, Belgium, October-November 2018. Association for Computatio… find
  20. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL ’02, pp. 311–318, USA, 2002. Association for Computational Linguistics. doi: 10.311… our card
  21. David Premack and Guy Woodruff. Does the chimpanzee have a theory of mind? Behavioral and brain sciences, 1(4):515–526, 1978. find
  22. Neil Rabinowitz, Frank Perbet, Francis Song, Chiyuan Zhang, SM Ali Eslami, and Matthew Botvinick. Machine theory of mind. In International conference on machine learning, pp. 4218– 4227. PMLR, 2018. find
  23. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019. find
  24. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. URL https://arxiv.org/abs/2103.00020. ↗ our card
  25. Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2019. URL https://arxiv.org/abs/1908.10084. ↗ our card
  26. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017. URL https://arxiv.org/abs/1707.06347. ↗
  27. Linda Smith and Chen Yu. Infants rapidly learn word-referent mappings via cross-situational statistics. Cognition, 106(3):1558–1568, 2008. ISSN 0010-0277. doi: https://doi.org/ 10.1016/j.cognition.2007.06.010. URL https://www.sciencedirect.com/science/ article/pii/S0010027707001795. — [doi:10.1016/j.cognition.2007.06.010](https://doi.org/10.1016/j.cognition.2007.06.010) find
  28. Michael Tomasello. Constructing a Language: A Usage-Based Theory of Language Acquisition, volume 1. Harvard University Press, 2005. find
  29. Ramakrishna Vedantam, Samy Bengio, Kevin Murphy, Devi Parikh, and Gal Chechik. Context-aware captions from context-agnostic supervision. CoRR, abs/1701.02870, 2017. URL http://arxiv.org/abs/1701.02870. ↗
  30. Chen Yu and Dana H. Ballard. A unified model of early word learning: Integrating statistical and social cues. Neurocomputing, 70(13):2149–2165, 2007. ISSN 0925-2312. doi: https://doi.org/ 10.1016/j.neucom.2006.01.034. URL https://www.sciencedirect.com/science/ article/pii/S092523120600508X. Selected… — [doi:10.1016/j.neucom.2006.01.034](https://doi.org/10.1016/j.neucom.2006.01.034) find
  31. Chen Yu and Linda B. Smith. Embodied attention and word learning by toddlers. Cognition, 125(2):244–262, 2012. ISSN 0010-0277. doi: https://doi.org/10.1016/j.cognition. 2012.06.016. URL https://www.sciencedirect.com/science/article/pii/ S0010027712001369. — [doi:10.1016/j.cognition](https://doi.org/10.1016/j.cognition) ↗
  32. Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou. When and why vision-language models behave like bags-of-words, and what to do about it?, 2022. URL https://arxiv.org/abs/2210.01936. — [arXiv:2210.01936](https://arxiv.org/abs/2210.01936) ↗
  33. Hao Zhu, Graham Neubig, and Yonatan Bisk. Few-shot language coordination by modeling theory of mind. In International Conference on Machine Learning, pp. 12901–12911. PMLR, 2021. find
  34. Hao Zhu, Yonatan Bisk, and Graham Neubig. Simulated language learning through communicative goals and linguistic input. Proceedings of the Annual Meeting of the Cognitive Science Society, 44:1351–1358, 2022. find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/liu-2023-computational-language-acquisition-theory-mind/