Elamur Research⌕
☰
home / papers / 2015 / A Bayesian Model of Grounded Color Semantics

A Bayesian Model of Grounded Color Semantics

Paper · 2015 · Brian McMahan and Matthew Stone

Summary

People name colors far less consistently than English's small set of basic color terms would suggest. In crowdsourced data, a description still carries about 3.845 bits of uncertainty once its color is known. McMahan and Stone therefore argue that grounded semantics should not reduce each word to one fixed classifier. Their Bayesian model treats a color term as a distribution over box-shaped categories in HSV space. The category boundaries are uncertain and are settled each time a speaker uses the word. A separate per-term "availability" parameter captures how readily speakers choose a term when it applies. The model is trained on a subset of Randall Munroe's 2010 crowdsourced color-naming corpus. In statistical evaluation, it predicts the data better than a brute-force memorization model and seems to generalize more sensibly to unseen colors. It also explains speaker choice as well as special-purpose models while still supporting broader semantic reasoning. The result is released as LUX, a resource for dialogue systems that must produce or interpret color references. The authors also see it as a template for vague meanings in other continuous domains.

Key points

  • LUX (Lexicon of Uncertain Color Standards) maps 829 distinct English color descriptions to probabilistic, potentially context-sensitive regions in HSV color space. Data, models and visualization tools are available at mcmahan.io/lux.
  • Training data: a subset of Munroe's (2010) corpus of about 3.4 million free-text color-patch descriptions.
  • Variability: the average conditional entropy of a description given its color is 3.845 bits, roughly equivalent to rolling a 14-sided die.
  • Model structure: each term has uncertain category thresholds (semantics) plus an availability weight α_k (choice). Speakers are assumed to say a term only when it applies, picking among applicable terms without aiming for maximal informativeness, in contrast to Smith et al. (2013).
  • Theoretical consequence: the hearer's pragmatic posterior equals the semantic posterior, so the model yields no implicatures. The authors interpret this as semantics being a Nash equilibrium.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Rutgers, The State University of New JerseyNew Brunswick · United States40 works · 12,486 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksPsychology · 1,649 worksExperimental and Cognitive Psychology · 930 works

Cite

Cite as Brian McMahan and Matthew Stone. 2015. A Bayesian Model of Grounded Color Semantics. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl_a_00126

@article{mcmahan2015bayesian,
  title = {{A Bayesian Model of Grounded Color Semantics}},
  author = {Brian McMahan and Matthew Stone},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2015},
  doi = {10.1162/tacl\_a\_00126},
  url = {https://doi.org/10.1162/tacl_a_00126}
}

References

49 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Hirotugu Akaike. 1974. A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6):716–723. find
  2. John R. Anderson. 1991. The adaptive nature of human categorization. Psychological Review, 98(3):409. find
  3. Jacob Andreas and Dan Klein. 2014. Grounding language with points and paths in continuous spaces. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning, pages 58–67, June. find
  4. Chris Barker. 2002. The dynamics of vagueness. Linguistics and Philosophy, 25(1):1–36. find
  5. Brent Berlin. 1991. Basic Color Terms: Their Universality and Evolution. Univ of California Press. find
  6. Susan E. Brennan and Herbert H. Clark. 1996. Conceptual pacts and lexical choice in conversation. Journal of Experimental Psychology: Learning, Memory and Cognition, 22(6):1482–1493. find
  7. Stephen P. Brooks and Andrew Gelman. 1998. General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics, 7(4):434–455. find
  8. Peter F. Brown, Vincent J. Della Pietra, Robert L. Mercer, Stephen A. Della Pietra, and Jennifer C. Lai. 1992. An estimate of an upper bound for the entropy of English. Computational Linguistics, 18(1):31–40.
  9. Elia Bruni, Gemma Boleda, Marco Baroni, and Nam-Khanh Tran. 2012. Distributional semantics in technicolor. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, pages 136–145.
  10. Stanley F. Chen and Joshua Goodman. 1996. An empirical study of smoothing techniques for language modeling. In Proceedings of the 34th annual meeting on Association for Computational Linguistics, pages 310– 318.
  11. David L. Chen and Raymond J. Mooney. 2008. Learning to sportscast: a test of grounded language acquisition. find
  12. In ICML ’08: Proceedings of the 25th international conference on Machine learning, pages 128–135. Siddhartha Chib and Edward Greenberg. 1995. Understanding the Metropolis–Hastings algorithm. The American Statistician, 49(4):327–335. find
  13. Jason Chuang, Maureen Stone, and Pat Hanrahan. 2008. A probabilistic model of the categorical association between colors. In Color Imaging Conference, pages 6–11. find
  14. Sam Cumming. 2013. Coordination and content. Philosophers’ Imprint, 13(4):1–16. Colin R. Dawson, Jeremy Wright, Antons Rebguns, Marco Valenzuela Escárcega, Daniel Fried, and Paul R. Cohen. 2013. A generative probabilistic framework for learning spatial language. In 2013 IEEE Third Joint Internation… find
  15. IEEE. David DeVault, Iris Oved, and Matthew Stone. 2006. Societal grounding is essential to meaningful language use. In Proceedings of the Twenty-first National Conference on Artificial Intelligence, pages 747–754. find
  16. Mark D. Fairchild. 2013. Color Appearance Models. The Wiley-IS&T Series in Imaging Science and Technology. find
  17. Wiley. Delia Graff Fara. 2000. Shifting sands: An interest-relative theory of vagueness. Philosophical Topics, 28(1):45–81. find
  18. Ali Farhadi, Ian Endres, Derek Hoiem, and David Forsyth. 2009. Describing objects by their attributes. 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 1778–1785, June. find
  19. George W. Furnas, Thomas K. Landauer, Louis M. Gomez, and Susan T. Dumais. 1987. The vocabulary problem in human-system communication. Communications of the ACM, 30(11):964–971. Peter Gärdenfors. 2000. find
  20. Conceptual Spaces. MIT Press. Andrew Gelman, Gareth O. Roberts, and Walter R. Gilks. 1996. Efficient Metropolis jumping rules. In J. M. Bernardo, J. O. Berger, A. P. Dawid, and A. F. Smith, editors, Bayesian Statistics 5, pages 599–607. find
  21. Oxford University Press. Herbert P. Grice. 1975. Logic and conversation. In P. Cole and J. Morgan, editors, Syntax and Semantics III: Speech Acts, pages 41–58. find
  22. Academic Press. Stevan Harnad. 1990. The symbol grounding problem. Physica D: Nonlinear Phenomena, 42(1–3):335–346. find
  23. Jeffrey Heer and Maureen Stone. 2012. Color naming models for color selection, image editing and palette design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 1007– 1016. find
  24. John F. Hughes, Andries van Dam, Morgan McGuire, David F. Sklar, James D. Foley, Steven K. Feiner, and Kurt Akeley. 2013. Computer Graphics: Principles and Practice (3rd Edition). Addison-Wesley Professional. Gerhard Jäger. 2010. Natural color categories are convex sets. In Maria Aloni, Harald Bast… find
  25. Springer. Fred Jelinek, Robert L. Mercer, Lalit R. Bahl, and James K. Baker. 1977. Perplexity–a measure of the difficulty of speech recognition tasks. The Journal of the Acoustical Society of America, 62:S63. find
  26. Paul Kay, Brent Berlin, Luisa Maffi, William R. Merri-field, and Richard Cook. 2009. find
  27. The World Color Survey. CSLI. Emiel Krahmer and Kees van Deemter. 2012. Computational generation of referring expressions: A survey. Computational Linguistics, 38(1):173–218.
  28. Jayant Krishnamurthy and Thomas Kollar. 2013. Jointly learning to parse and perceive: Connecting natural language to the physical world. Transactions of the Association for Computational Linguistics, 1(2):193–206.
  29. Alice Kyburg and Michael Morreau. 2000. Fitting words: Vague words in context. Linguistics and Philosophy, 23(6):577–597. find
  30. Johan Maurice Gisele Lammens. 1994. A computational model of color perception and color naming. Ph.D. thesis, SUNY Buffalo. find
  31. Staffan Larsson. 2013. Formal semantics for perceptual classification. Journal of Logic and Computation. Advance online publication. doi: 10.1093/log-com/ext059. — [doi:10.1093/log-com/ext059](https://doi.org/10.1093/log-com/ext059) find
  32. Daniel Lassiter. 2009. Vagueness as probabilistic linguistic knowledge. In Rick Nouwen, Robert van Rooij, Uli Sauerland, and Hans-Christian Schmitz, editors, Vagueness in Communication - International Workshop, ViC 2009, held as part of ESSLLI 2009, Bordeaux, France, July 20-24, 2009. Revised Select… find
  33. Springer. David K. Lewis. 1969. find
  34. Convention: A Philosophical Study. Harvard University Press, Cambridge, MA. Cynthia Matuszek, Nicholas Fitzgerald, Luke Zettlemoyer, Liefeng Bo, and Dieter Fox. 2012. A joint model of language and perception for grounded attribute learning. In Proceedings of the 29th International Conference on Mach… find
  35. Nikolaos Mavridis and Deb Roy. 2006. Grounded situation models for robots: Where words and percepts meet. In Intelligent Robots and Systems, 2006 IEEE/RSJ International Conference on, pages 4690– 4697. find
  36. IEEE. Timothy Meo, Brian McMahan, and Matthew Stone. 2014. Generating and resolving vague color references. In SEMDIAL 2014: THE 18th Workshop on the Semantics and Pragmatics of Dialogue, pages 107– 115. find
  37. Nicholas Metropolis, Arianna W. Rosenbluth, Marshall N. Rosenbluth, Augusta H. Teller, and Edward Teller. 1953. Equation of state calculations by fast computing machines. The Journal of Chemical Physics, 21(6):1087–1092. find
  38. Randall Munroe. 2010. Color survey results. Online at http://blog.xkcd.com/2010/05/03/color-survey-results/. ↗
  39. Kimele Persaud and Pernille Hemmer. 2014. The influence of knowledge and expectations for color on episodic memory. In P Bello, M Guarini, M Mc-Shane, and B Scassellati, editors, Proceedings of the 36th Annual Conference of the Cognitive Science Society, pages 1162–1167. find
  40. Terry Regier, Paul Kay, and Richard S. Cook. 2005. Focal colors are universal after all. Proceedings of the National Academy of Sciences, 102:8386–8391. find
  41. Terry Regier, Paul Kay, and Naveen Khetarpal. 2007. Color naming reflects optimal partitions of color space. Proceedings of the National Academy of Sciences, 104:1436–1441. find
  42. Carina Silberer, Vittorio Ferrari, and Mirella Lapata. 2013. Models of Semantic Representation with Visual Attributes. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 572–582. find
  43. Nathaniel J. Smith, Noah D. Goodman, and Michael C. Frank. 2013. Learning and using language via recursive pragmatic reasoning about other agents. In Advances in Neural Information Processing Systems 26, pages 3039–3047. find
  44. Stefanie Tellex, Thomas Kollar, and Steven Dickerson. 2011a. Approaching the symbol grounding problem with probabilistic graphical models. AI magazine, 32(4):64–76. find
  45. Stefanie Tellex, Thomas Kollar, Steven Dickerson, Matthew R Walter, Ashis Gopal Banerjee, Seth J Teller, and Nicholas Roy. 2011b. Understanding natural language commands for robotic navigation and mobile manipulation. In Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence, pag… find
  46. Terry Winograd. 1970. Procedures as a representation for data in a computer program for understanding natural language. Ph.D. thesis, MIT. find
  47. Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. 2014. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics, 2:67–78. find
  48. John M. Zelle and Raymond J. Mooney. 1996. Learning to parse database queries using inductive logic programming. In Proceedings of the National Conference on Artificial Intelligence, pages 1050–1055.
  49. Luke S. Zettlemoyer and Michael Collins. 2005. Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. In UAI ’05, Proceedings of the 21st Conference in Uncertainty in Artificial Intelligence, pages 658–666.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mcmahan-2015-bayesian-model-grounded-color-semantics/