Elamur Research⌕
☰
home / papers / 2020 / Continual Adaptation for Efficient Machine Communication

Continual Adaptation for Efficient Machine Communication

Paper · 2020 · Robert Hawkins, Minae Kwon, Dorsa Sadigh et al.

Summary

Neural language models can reproduce the conventions in their training data, but unlike people they cannot invent new shorthand with a specific partner during an interaction. Hawkins and colleagues treat this kind of ad hoc convention formation as online domain adaptation. An agent starts from a generic pre-trained language model and keeps inferring the language model of its current partner. They frame this as Bayesian inference: a prior shared across speakers is combined with a likelihood based on observed language use.

To test the idea, they introduce a benchmark built on natural images, a repeated reference game in which a speaker and a listener refer to the same targets again and again. After every trial, the algorithm takes a few gradient steps. It combines three parts:

  • a contrastive loss that includes pragmatic reasoning,
  • KL regularization toward the original model,
  • data augmentation that assigns credit to fragments of utterances.

In COCO simulations and live games with human partners, the adapted agents became more accurate and needed fewer words over time. Ablations show that each component is necessary. The work matters for dialogue systems that should personalize to users without losing general language knowledge.

Key points

  • Benchmark: speaker and listener see a shared set of images (e.g., four cat photos). The speaker describes a privately marked target, the listener outputs a softmax over the images, and both receive feedback. Each target appears six times, so changes in descriptions can be tracked across the dialogue history.
  • Algorithm components: (i) a pragmatics-aware contrastive objective, (ii) KL regularization that prevents overfitting to sparse partner data and catastrophic forgetting, and (iii) compositional data augmentation that links later short utterances to earlier long ones.
  • Evaluation: simulations on COCO plus real-time reference games with naive human partners. The base model is a canonical CNN-RNN image captioning architecture, and the full RNN module is adapted end-to-end.
  • Future directions: a MAML-style meta-learning outer loop for updating the shared prior Θ, low-rank context-dependent weight adaptation (following Jaech and Ostendorf), and richer feedback such as clarification questions.
  • Publication and code: CoNLL 2020, pp. 408–419; code and materials at github.com/hawkrobe/continual-adaptation.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Stanford UniversityStanford · United States434 works · 314,419 citations🎓Princeton UniversityPrinceton · United States90 works · 39,109 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Robert Hawkins, Minae Kwon, Dorsa Sadigh and Noah Goodman. 2020. Continual Adaptation for Efficient Machine Communication. Proceedings of the 24th Conference on Computational Natural Language Learning.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2020.conll-1.33

@inproceedings{hawkins2020continual,
  title = {{Continual Adaptation for Efficient Machine Communication}},
  author = {Robert Hawkins, Minae Kwon, Dorsa Sadigh and Noah Goodman},
  booktitle = {Proceedings of the 24th Conference on Computational Natural Language Learning},
  year = {2020},
  doi = {10.18653/v1/2020.conll-1.33},
  url = {https://doi.org/10.18653/v1/2020.conll-1.33}
}

References

62 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jacob Andreas and Dan Klein. 2016. Reasoning about pragmatics with neural listeners and speakers. In Proceedings of EMNLP, pages 1173–1182.
  2. David DeVault and Matthew Stone. 2009. Learning to interpret utterances using dialogue history. In Proceedings of EACL, pages 184–192. find
  3. Jerome R Bellegarda. 2004. Statistical language model adaptation: review and perspectives. Speech Communication, 42(1):93–108. find
  4. Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. 2010. A theory of learning from different domains. Machine learning, 79(1-2):151–175. find
  5. Mark Dingemanse, Seán G Roberts, Julija Baranova, Joe Blythe, Paul Drew, Simeon Floyd, Rosa S Gisladottir, Kobin H Kendrick, Stephen C Levinson, Elizabeth Manrique, Giovanni Rossi, and N. J. Enfield. 2015. Universal principles in the repair of communication problems. PloS one, 10(9):e0136100. find
  6. Leon Bergen, Roger Levy, and Noah Goodman. 2016. Pragmatic reasoning through semantic inference. Semantics and Pragmatics, 9(20). find
  7. Paul Drew. 1997. ‘Open’ class repair initiators in response to sequential sources of troubles in conversation. Journal of Pragmatics, 28(1):69–101. find
  8. John Blitzer, Mark Dredze, and Fernando Pereira. 2007. Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In Proceedings of ACL, pages 440–447.
  9. Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning, pages 1126–1135.
  10. Holly P Branigan, Martin J Pickering, Jamie Pearson, Janet F McLean, and Ash Brown. 2011. The role of beliefs in lexical alignment: Evidence from dialogs with humans and computers. Cognition, 121(1):41– 57. find
  11. Michael C Frank, Noah D Goodman, and Joshua B Tenenbaum. 2009. Using speakers’ referential intentions to model early cross-situational word learning. Psychological Science, 20(5):578–585. find
  12. Xinlei Chen, Hao Fang, Tsung-Yi Lin, Ramakrishna Vedantam, Saurabh Gupta, Piotr Dollár, and C Lawrence Zitnick. 2015. Microsoft COCO captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325.
  13. Maxime Chevalier-Boisvert, Dzmitry Bahdanau, Salem Lahlou, Lucas Willems, Chitwan Saharia, Thien Huu Nguyen, and Yoshua Bengio. 2019. BabyAI: First steps towards grounded language learning with a human in the loop. In Proceedings Daniel Fried, Jacob Andreas, and Dan Klein. 2018. Unified pragmatic mo…
  14. Alexandre Galashov, Siddhant M Jayakumar, Leonard Hasenclever, Dhruva Tirumala, Jonathan Schwarz, Guillaume Desjardins, Wojciech M Czarnecki, Yee Whye Teh, Razvan Pascanu, and Nicolas Heess. 2018. Information asymmetry in KL-regularized RL. In Proceedings of the 7th International Conference on Learn… find
  15. Xavier Glorot, Antoine Bordes, and Yoshua Bengio. 2011. Domain adaptation for large-scale sentiment classification: A deep learning approach. In Proceedings of the 28th International Conference on Machine Learning, pages 513–520. find
  16. Noah D Goodman and Michael C Frank. 2016. Pragmatic language interpretation as probabilistic inference. Trends in Cognitive Sciences, 20(11):818 – 829. find
  17. Erin Grant, Chelsea Finn, Sergey Levine, Trevor Darrell, and Thomas Griffiths. 2018. Recasting gradient-based meta-learning as hierarchical bayes. In Proceedings of the 6th International Conference on Learning Representations. find
  18. H. P. Grice. 1975. Logic and conversation. In P. Cole and J. Morgan, editors, Syntax and Semantics, pages 43–58. Academic Press, New York. find
  19. Daniel Grodner and Julie C Sedivy. 2011. The effect of speaker-specific information on pragmatic inferences. In The processing and acquisition of reference, volume 2327, pages 239–272. MIT Press. find
  20. Janosch Haber, Tim Baumgärtner, Ece Takmaz, Lieke Gelderloos, Elia Bruni, and Raquel Fernández. 2019. The PhotoBook dataset: Building common ground through visually-grounded dialogue. In Proceedings of ACL, pages 1895–1910.
  21. Robert D Hawkins, Michael C Frank, and Noah D Goodman. 2017. Convention-formation in iterated reference games. In Proceedings of the 39th annual meeting of the Cognitive Science Society. find
  22. Robert D Hawkins, Michael C Frank, and Noah D Goodman. 2020a. Characterizing the dynamics of learning in repeated reference games. Cognitive Science, 44(6):e12845.
  23. Robert D Hawkins, Noah D Goodman, Adele E Goldberg, and Thomas L Griffiths. 2020b. Generalizing meanings from partners to populations: Hierarchical inference supports convention formation on networks. In Proceedings of the 42nd annual meeting of the Cognitive Science Society. find
  24. Aaron Jaech and Mari Ostendorf. 2018a. Low-rank RNN adaptation for context-aware language modeling. Transactions of the Association for Computational Linguistics, 6:497–510. find
  25. Aaron Jaech and Mari Ostendorf. 2018b. Personalized language model for query auto-completion. In Proceedings of ACL, pages 700–705. find
  26. Dave F Kleinschmidt and T Florian Jaeger. 2015. Robust speech perception: Recognize the familiar, generalize to the similar, and adapt to the novel. Psychological Review, 122(2):148. find
  27. Reinhard Kneser and Volker Steinbiss. 1993. On the dynamic adaptation of stochastic language models. In IEEE International Conference on Acoustics, Speech, and Signal Processing, pages 586–589. find
  28. Theodora Koulouri, Stanislao Lauria, and Robert D Macredie. 2016. Do (and say) as I say: Linguistic adaptation in human–computer dialogs. Human– Computer Interaction, 31(1):59–95. find
  29. Ben Krause, Emmanuel Kahembwe, Iain Murray, and Steve Renals. 2017. Dynamic evaluation of neural sequence models. In Proceedings of the 35th International Conference on Machine Learning, pages 2771–2780. find
  30. Robert M Krauss and Sidney Weinheimer. 1964. Changes in reference phrases as a function of frequency of usage in social interaction: A preliminary study. Psychonomic Science, 1(1-12):113–114. find
  31. Angeliki Lazaridou, Alexander Peysakhovich, and Marco Baroni. 2016. Multi-agent cooperation and the emergence of (natural) language. In Proceedings of the 5th International Conference on Learning Representations. find
  32. David Lewis. 1969. Convention: A philosophical study. Harvard University Press. find
  33. Jiwei Li, Alexander H Miller, Sumit Chopra, Marc’Aurelio Ranzato, and Jason Weston. 2016. Learning through dialogue interactions by asking questions. In Proceedings of the 5th International Conference on Learning Representations. find
  34. Xiao Li and Jeff Bilmes. 2007. A Bayesian divergence prior for classifier adaptation. In Artificial Intelligence and Statistics, pages 275–282. find
  35. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft COCO: Common objects in context. In Proceedings of ECCV, pages 740–755. Springer.
  36. Chaojun Liu, Yongqiang Wang, Kshitiz Kumar, and Yifan Gong. 2016. Investigations on speaker adaptation of LSTM RNN models for speech recognition. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pages 5020–5024. find
  37. Arthur B Markman and Valerie S Makin. 1998. Referential communication and category acquisition. Journal of Experimental Psychology: General, 127(4):331. find
  38. David Marr. 2010. Vision: A computational investigation into the human representation and processing of visual information. MIT press. find
  39. Charles Metzing and Susan E Brennan. 2003. When conceptual pacts are broken: Partner-specific effects on the comprehension of referring expressions. Journal of Memory and Language, 49(2):201–213. find
  40. Yajie Miao and Florian Metze. 2015. On speaker adaptation of long short-term memory recurrent neural networks. In Proceedings of the 16th Annual Conference of the International Speech Communication Association, pages 1101–1105. find
  41. Tomáš Mikolov, Martin Karafiát, Lukáš Burget, Jan Černockỳ, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In Proceedings of the 11th annual conference of the International Speech Communication Association, pages 1045–1048. find
  42. Ian Stewart, Diyi Yang, and Jacob Eisenstein. 2020. Characterizing collective attention via descriptor context: A case study of public discussions of crisis events. In Proceedings of the 14th International AAAI Conference on Web and Social Media, pages 650–660. find
  43. Will Monroe, Robert D. Hawkins, Noah D. Goodman, and Christopher Potts. 2017. Colors in context: A pragmatic neural model for grounded language understanding. Transactions of the Association for Computational Linguistics, 5:325–338.
  44. Marten Van Schijndel and Tal Linzen. 2018. A neural model of adaptation in reading. In Proceedings of EMNLP, pages 4704–4710. find
  45. Anusha Nagabandi, Chelsea Finn, and Sergey Levine. 2019. Deep online learning via meta-learning: Continual adaptation for model-based RL. In Proceedings of the 7th International Conference on Learning Representations. find
  46. Yasuhito Ohsugi, Itsumi Saito, Kyosuke Nishida, Hisako Asano, and Junji Tomita. 2019. A simple but effective method to incorporate multi-turn context with BERT for conversational machine comprehension. In Proceedings of the First Workshop on NLP for Conversational AI, pages 11–17. find
  47. Naho Orita, Eliana Vornov, Naomi Feldman, and Hal Daumé III. 2015. Why discourse affects speakers’ choice of referring expressions. In Proceedings of ACL, pages 1639–1649. find
  48. Ramakrishna Vedantam, Samy Bengio, Kevin Murphy, Devi Parikh, and Gal Chechik. 2017. Context-aware captions from context-agnostic supervision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1070–1079. find
  49. Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2015. Show and tell: A neural image caption generator. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3156–3164.
  50. Sida I Wang, Samuel Ginn, Percy Liang, and Christoper D Manning. 2017. Naturalizing a programming language via interactive learning. In Proceedings of ACL, pages 929–938. find
  51. Sida I Wang, Percy Liang, and Christopher D Manning. 2016. Learning language games through interaction. In Proceedings of ACL, page 2368–2378. find
  52. Giuseppe Riccardi and Allen L Gorin. 2000. Stochastic language adaptation over time and state in natural spoken dialog systems. IEEE Transactions on Speech and Audio Processing, 8(1):3–10. find
  53. Deanna Wilkes-Gibbs and Herbert H Clark. 1992. Coordinating beliefs in conversation. Journal of Memory and Language, 31(2):183–194. find
  54. Anthony Robins. 1995. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Science, 7(2):123–146. find
  55. Ilker Yildirim, Judith Degen, Michael K Tanenhaus, and T Florian Jaeger. 2016. Talker-specificity and adaptation in quantifier interpretation. Journal of Memory and Language, 87:128–143. find
  56. Timo B Roettger and Michael Franke. 2019. Evidential strength of intonational cues and rational adaptation to (un-) reliable intonation. Cognitive Science, 43(7):e12745. find
  57. Subhro Roy, Michael Noseworthy, Rohan Paul, Dae-hyung Park, and Nicholas Roy. 2019. Leveraging past references for robust language grounding. In Proceedings of CoNLL, pages 430–440. find
  58. Sebastian Schuster and Judith Degen. 2020. I know what you’re probably going to say: Listener adaptation to variable use of uncertainty expressions. Cognition, 203:104285. find
  59. Nathaniel J Smith, Noah Goodman, and Michael Frank. 2013. Learning and using language via recursive pragmatic reasoning about other agents. In Advances in Neural Information Processing Systems, pages 3039–3047. find
  60. Ieva Staliūnaitė, Hannah Rohde, Bonnie Webber, and Annie Louis. 2018. Getting to “Hearer-old”: Charting referring expressions across time. In Proceedings of EMNLP, pages 4350–4359. find
  61. Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. 2014. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics, 2:67–78. find
  62. Dong Yu, Kaisheng Yao, Hang Su, Gang Li, and Frank Seide. 2013. KL-divergence regularized deep neural network adaptation for improved large vocabulary speech recognition. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pages 7893–7897. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hawkins-2020-continual-adaptation-efficient-machine-communication/