Elamur Research⌕
☰
home / papers / 2018 / A formal semantics for situated conversation

A formal semantics for situated conversation

Paper · 2018 · Julie Hunter, Nicholas Asher and Alex Lascarides

Summary

Formal work on how non-verbal context shapes meaning usually treats that context as a source of referents for deictic or anaphoric expressions. This paper studies a different case. A nonlinguistic event, such as a nod toward scratches on a wall, can supply an entire discourse unit that enters coherence relations with spoken utterances, even though no word points to it. The authors use constructed examples (Scratches, Table) and naturally occurring data from the Settlers corpus, where game events act as nonlinguistic moves inside players' conversations. They argue that coherence-based theories, specifically SDRT, should let event-derived units serve as arguments of relations like Explanation and Background. They also show this is not a cosmetic change. It forces a rethink of what nonlinguistic context and rhetorical relations are, and it requires new salience and attachment constraints, because situated data show asymmetric and interleaved structures. It also needs a dynamic model theory in which events join the interpreted structure while also changing the world of evaluation. The result lays groundwork for parsing situated, multimodal dialogue.

Key points

  • The central hypothesis (NDU) is that nonlinguistic events can contribute whole discourse units. In Scratches, a label ε standing for the inferred wall-scratching event takes the place of a clause, giving Explanation(π1, ε) and Background(ε, π3).
  • Leaving out the inferred event has visible effects. In Scratches, the cooking wrongly overlaps the punishment in time. In Table, the discourse becomes barely coherent.
  • The Settlers corpus offers a stable shared context across many moves, which lets the authors set aside the conceptualization problem. It shows that constraints on hierarchical discourse development differ from those of purely linguistic discourse.
  • The approach is set in SDRT, where the semantics/pragmatics interface sees only a partial description of content, not linguistic form. It is contrasted with Ginzburg's QUD, which encodes speech-act information in the grammar.
  • Future work includes a perceptual module to individuate and conceptualize events, and extending statistical discourse parsers (Muller et al. 2012) to situated units.

Interesting ideas

  • I1. What triggers the search for an unstated unit is not a word but the failure to find a coherent relation between adjacent moves; the gap itself licenses positing a missing event.
  • I2. Expectations about the discourse relation (e.g., that an Explanation is coming) can shape how an ambiguous unit is conceptualized, so inference runs top-down as well as bottom-up.
  • I3. An omission test shows an event's role: remove it and check whether temporal ordering shifts or coherence collapses.
  • I4. The pragmatic layer reads only partial content descriptions, not surface form, so units from text and units from non-text sources can enter the same relations.
  • I5. A narrated or observed event both adds a unit to the discourse structure and updates the world against which later utterances are evaluated.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): when two clauses get no plausible discourse relation, the system could hypothesize an implicit bridging event and record that hypothesis explicitly instead of forcing a weak relation.
  • I2 (frames): an expected relation type could be used to rank competing frame readings of the next clause, with the expectation logged as the reason for the choice.
  • I3 (learn): this suggests a negative control in which a bridging event is deleted from the world of the text, and answers that stay unchanged expose reasoning that never used it.
  • I4 (world): facts from compiled knowledge or the world state could be wrapped as discourse units of the same type as clause units, so relations can link them uniformly.
  • I5 (world): event order and states could be updated whenever an event unit is attached, so later clauses are checked against the changed world rather than the initial one.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of EdinburghEdinburgh · United Kingdom255 works · 68,771 citations🔬Centre National de la Recherche ScientifiqueParis · France104 works · 60,877 citations🎓Universitat Pompeu FabraBarcelona · Spain36 works · 11,433 citations🔬Institut de Recherche en Informatique de ToulouseToulouse · France23 works · 1,667 citations

📚 Shelves 3The field of science this work belongs to (OpenAlex fields).Arts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 worksPhilosophy · 211 works

🧭 Topics 1Research subjects and languages the work is about.Discourse analysis · 394 works

Cite

Cite as Julie Hunter, Nicholas Asher and Alex Lascarides. 2018. A formal semantics for situated conversation. Semantics and Pragmatics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3765/sp.11.10

@article{hunter2018formal,
  title = {{A formal semantics for situated conversation}},
  author = {Julie Hunter, Nicholas Asher and Alex Lascarides},
  journal = {Semantics and Pragmatics},
  year = {2018},
  doi = {10.3765/sp.11.10},
  url = {https://doi.org/10.3765/sp.11.10}
}

References

61 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Afantenos, Stergos & Nicholas Asher. 2010. Testing SDRT’s right frontier. In Chu-Ren Huang & Dan Jurafsky (eds.), The 23rd international conference on computational linguistics (COLING), 1–9. Beijing: COLING 2010 Organizing Committee. http://www.aclweb.org/anthology/C10-1001. ↗
  2. Afantenos, Stergos, Nicholas Asher, Farah Benamara, Anaïs Cadilhac, Cedric Dégremont, Pascal Denis, Markus Guhe, Simon Keizer, Alex Lascarides, Oliver Lemon, Philippe Muller, Soumya Paul, Vincent Popescu, Verena Rieser & Laure Vieu. 2012. Modelling strategic conversation: Model, annotation design and corpus. In Sarah Brown-Schmidt, Jonathan Ginzburg & A formal semantics for situated conversation Staffan Larsson (eds.), The 16th workshop on the semantics and pragmatics of dialogue (SeineDial), 145–146. Paris. https://hal.archives-ouvertes.f r/hal-01138035/document. ↗
  3. Alahverdzhieva, Katya & Alex Lascarides. 2010. Analysing language and co-verbal gesture in constraint-based grammars. The 17th International Conference on Head-Driven Phrase Structure Grammar (HPSG). 5–25. https://doi.org/10.21248/hpsg.2010.1. ↗
  4. Asher, Nicholas. 1993. Reference to abstract objects in discourse. Dordrecht: Kluwer Academic Publishers. https://doi.org/10.1007/978-94-011-1715-9. ↗
  5. Asher, Nicholas, Julie Hunter, Mathieu Morey, Farah Benamara & Stergos Afantenos. 2016. Discourse structure and dialogue acts in multiparty dialogue: The STAC corpus. In Nicoletta Calzolari (Conference Chair), Khalid Choukri, Thierry Declerck, Sara Goggi, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Helene Mazo, Asuncion Moreno, Jan Odijk & Stelios Piperidis (eds.), The tenth international conference on language resources and evaluation (LREC 2016), 2721–2727. Portorož: European Language Resources Association (ELRA). http://www.lrec-conf.org/proceedings/lrec2016/summaries/339.html. ↗
  6. Asher, Nicholas & Alex Lascarides. 2003. Logics of conversation. New York, NY: Cambridge University Press. find
  7. Asher, Nicholas, Soumya Paul & Antoine Venant. 2017. Message exchange games in strategic conversations. Journal of Philosophical Logic 46(4). 355–404. https://doi.org/10.1007/s10992-016-9402-1. ↗
  8. Baroni, Marco. 2016. Grounding distributional semantics in the visual world. Language and Linguistics Compass 10(1). 3–13. https://doi.org/10.1111/l nc3.12170. ↗
  9. Chambers, Nathanial, James Allen, Lucian Galescu & Hyuckchul Jung. 2005. A dialogue-based approach to multi-robot team control. In Lynne E. Parker, Frank E. Schneider & Alan C. Schultz (eds.), Multi-robot systems. From swarms to intelligent automata, volume III: Proceedings from the 2005 international workshop on multi-robot systems, 257–262. Washington, D.C. https://doi.org/10.1007/1-4020-3389-3_21. ↗
  10. Clarke, Alasdair, Micha Eisner & Hannah Rohde. 2015. Giving good directions: Order of mention reflects visual salience. Frontiers in Psychology 6(1793). 1–10. https://doi.org/10.3389/fpsyg.2015.01793. ↗
  11. Davidson, Donald. 1967. The logical form of action sentences. In Nicholas Rescher (ed.), The logic of decision and action, 81–95. Pittsburgh, PA: University of Pittsburgh Press. https://doi.org/10.2307/jj.13027259.6. ↗
  12. Hunter, Asher, and Lascarides Dobnik, Simon, Robin Cooper & Staffan Larsson. 2013. Modelling language, action, and perception in type theory with records. In Denys Duchier & Yannick Parmentier (eds.), Constraint solving and language processing (Lecture Notes in Computer Science 8114), 70–91. Berlin & Heidelberg: Springer. https://doi.org/10.1007/978-3-642-41578-4_5. ↗
  13. Dynel, Marta. 2010. Not hearing things — hearer/listener categories in polylogues. mediAzioni 9. http://www.mediazioni.sitlec.unibo.it/images/sto ries/PDF_folder/document-pdf/2010/dynel_2010.pdf. ↗
  14. Elbourne, Paul D. 2002. Situations and individuals. Cambridge, MA: Massachusetts Institute of Technology dissertation. Published in Elbourne 2005. http://hdl.handle.net/1721.1/8157. ↗
  15. Elbourne, Paul D. 2005. Situations and individuals. Cambridge, MA: Massachusetts Institute of Technology Press. Edited version of Elbourne 2002. find
  16. Elsner, Micha & Eugene Charniak. 2011. Disentangling chat with local coherence models. The 49th Annual Meeting of the Association for Computational Linguistics (ACL). 1179–1189. https://aclanthology.org/P11-1118. ↗
  17. Forbes, Maxwell, Rajesh Rao, Luke Zettlemoyer & Maia Cakmak. 2015. Robot programming by demonstration with situated spatial language understanding. The 2015 IEEE International Conference on Robotics and Automation (ICRA). 2014–2020. https://doi.org/10.1109/ICRA.2015.7139 462. ↗
  18. Foster, Mary Ellen & Ronald P. A. Petrick. 2014. Planning for social interaction with sensor uncertainty. In Gabriella Cortellessa, Mark Giuliano, Riccardo Rasconi & Neil Yorke-Smith (eds.), The ICAPS 2014 scheduling and planning applications workshop (SPARK), 19–20. Portsmouth, NH. https://res earchportal.hw.ac.uk/en/publications/planning-for-social-interaction-w ith-sensor-uncertainty. ↗
  19. Ginzburg, Jonathan. 2012. The interactive stance: Meaning for conversation. Oxford: Oxford University Press. https://doi.org/10.1093/acprof:oso/97 80199697922.001.0001. ↗
  20. Ginzburg, Jonathan & Raquel Fernández. 2005. Scaling up from dialogue to multilogue: Some principles and benchmarks. The 43rd Annual Meeting of the Association for Computational Linguistics (ACL). 231–238. https://d oi.org/10.3115/1219840.1219869. ↗
  21. Ginzburg, Jonathan & Ivan A. Sag. 2001. Interrogative investigations: The form, meaning and use of English interrogatives. Palo Alto, CA: CSLI. find
  22. Goffman, Erving. 1981. Forms of talk. Philadelphia, PA: University of Pennsylvania Press. A formal semantics for situated conversation Groenendijk, Jeroen. 2003. Questions and answers: Semantics and logic. In Raffaella Bernardi & Michael Moortgat (eds.), The 2nd CologNET-ElSNET symposium. Questions and answers: Theoretical and applied perspectives, 16–23. Utrecht. https://pure.uva.nl/ws/files/3604214/27248_groenendij kquestanansw.pdf. ↗
  23. Groenendijk, Joroen, Martin Stokhof & Frank Veltman. 1996. This might be it. In Jerry Seligman & Dag Westerståhl (eds.), Language, logic and computation: The 1994 Moraga proceedings, 255–270. Palo Alto, CA: CSLI. find
  24. Halliday, Michael A.K. 1967. Notes on transitivity and theme in English: Part 2. Journal of Linguistics 3(2). 199–244. https://doi.org/10.1017/S0022226 700016613. ↗
  25. Hobbs, Jerry R. 1979. Coherence and coreference. Cognitive Science 3(1). 67– 90. https://doi.org/10.1207/s15516709cog0301_4. ↗
  26. Hobbs, Jerry R. 1985. On the coherence and structure of discourse. Palo Alto, CA. find
  27. Hobbs, Jerry R., Martin Stickel, Douglas Appelt & Paul Martin. 1993. Interpretation as abduction. Artificial Intelligence 63(1–2). 69–142. https://doi.or g/10.1016/0004-3702(93)90015-4. ↗
  28. Hunter, Julie. 2014. Structured contexts and anaphoric dependencies. Philosophical Studies 168(1). 35–58. https://doi.org/10.1007/s11098-013-0209-4. ↗
  29. Hunter, Julie, Nicholas Asher & Alex Lascarides. 2015. Integrating non-linguistic events into discourse structure. In Matthew Purver, Mehrnoosh Sadrzadeh & Matthew Stone (eds.), The 11th international conference on computational semantics (IWCS), 184–194. London. http://www.aclweb.org/a nthology/W15-0123. ↗
  30. Joty, Shafiq, Giuseppe Carenini & Raymond Ng. 2015. CODRA: A novel discriminative framework for rhetorical analysis. Computational Linguistics 41(3). 385–435. https://doi.org/10.1162/COLI_a_00226. ↗
  31. Kamp, Hans. 1981. The paradox of the heap. In Uwe Mönnich (ed.), Aspects of philosophical logic, 225–277. Dordrecht: Springer. https://doi.org/10.100 7/978-94-009-8384-7. ↗
  32. Kamp, Hans. 1990. Prologomena to a structural theory of belief and other attitudes. In C. Anthony Anderson & Joseph Owens (eds.), Propositional attitudes: The role of content in logic, language and mind, 27–90. Palo Alto, CA: CSLI. find
  33. Hunter, Asher, and Lascarides Kaplan, David. 1989. Demonstratives. In Joseph Almog, John Perry & Howard Wettstein (eds.), Themes from Kaplan, 481–563. Oxford & New York, NY: Oxford University Press. find
  34. Kehler, Andrew. 2002. Coherence, reference and the theory of grammar. Palo Alto, CA: CSLI. find
  35. Kendon, Adam. 1983. Gesture and speech: How they interact. In John Wiemann & Randall Harrison (eds.), Nonverbal interaction, 13–46. Beverly Hills: Sage Publications. find
  36. Kranstedt, Alfred, Andy Lüking, Thies Pfeiffer, Hannes Rieser & Ipke Wach-smith. 2006. Deixis: How to determine demonstrated objects using a pointing cone. In Sylvie Gibet, Nicolas Courty & Jean-François Kamp (eds.), Gesture in human-computer interaction and simulation (GW 2005. Lecture Notes in Computer Science 3881), 300–311. Berlin & Heidelberg: Springer. https://doi.org/10.1007/11678816_34. ↗
  37. Larsson, Staffan. 2015. Formal semantics for perceptual classification. Journal of Logic and Computation 25(2). 335–369. https://doi.org/10.1093/lo gcom/ext059. ↗
  38. Lascarides, Alex & Nicholas Asher. 2009. Agreement, disputes and commitment in dialogue. Journal of Semantics 26(2). 109–158. https://doi.org/1 0.1093/jos/ffn013. ↗
  39. Lascarides, Alex & Matthew Stone. 2009. A formal semantic analysis of gesture. Journal of Semantics 26(4). 393–449. https://doi.org/10.1093/jos/f fp004. ↗
  40. Lepore, Ernest & Matthew Stone. 2014. Imagination and convention: Distinguishing grammar and inference in language. Oxford: Oxford University Press. https://doi.org/10.1093/acprof:oso/9780198717188.001.0001. ↗
  41. Liang, Percy. 2005. Semi-supervised learning for natural language. Cambridge, MA: Massachusetts Institute of Technology dissertation. https://doi.org/1721.1/33296. ↗
  42. Lücking, Andy. 2016. Modeling co-verbal gesture perception in type theory with records. 2016 Federated Conference on Computer Science and Information Systems (FedCSIS). 383–392. https://annals-csis.org/proceedings/2016/pliks/83.pdf. ↗
  43. Maier, Emar. 2009. Proper names and indexicals trigger rigid presuppositions. Journal of Semantics 26(3). 253–315. https://doi.org/10.1093/jos/ff p006. ↗
  44. Matuszek, Cynthia, Nicholas FitzGerald, Luke Zettlemoyer, Liefeng Bo & Dieter Fox. 2012. A joint model of language and perception for grounded A formal semantics for situated conversation attribute learning. In John Langford & Joelle Pineau (eds.), The 29th international conference on machine learning (ICML 2012). Edinburgh. https://arxiv.org/abs/1206.6423. ↗
  45. Moore, Johanna D. & Cécile L. Paris. 1993. Planning text for advisory dialogues: Capturing intentional and rhetorical information. Computational Linguistics 19(4). 651–695. https://dl.acm.org/doi/10.5555/972500.97250 4. ↗
  46. Muller, Philippe, Stergos Afantenos, Pascal Denis & Nicholas Asher. 2012. Constrained decoding for text-level discourse parsing. In Martin Kay & Christian Boitet (eds.), The 24th international conference in computational linguistics (COLING): Technical papers, 1883–1900. Mumbai: The COLING 2012 Organizing Committee. http://www.aclweb.org/anthology/C12-111 5. ↗
  47. Osborne, Martin & Ariel Rubinstein. 1990. Bargaining and markets. San Diego, CA: Academic Press. find
  48. Perzanowski, Dennis, Alan Schultz, William Adams, Elaine Marsh & Magda Bugajska. 2001. Building a multimodal human-robot interface. Intelligent Systems 16(1). 16–21. https://doi.org/10.1109/MIS.2001.1183338. ↗
  49. Polanyi, Livia. 1985. A theory of discourse structure and discourse coherence. Chicago Linguistics Society (CLS) 21(1). 306–322. find
  50. Roberts, Craige. 1996. Information structure in discourse: Towards an integrated formal theory of pragmatics, 1998 revision. In Jae Hak Yoon & Andreas Kathol (eds.), Papers in semantics (Working Papers in Linguistics 49). The Ohio State University, Department of Linguistics. A revised version was published in Roberts 2012.
  51. Roberts, Craige. 2012. Information structure in discourse: Towards an integrated formal theory of pragmatics. Semantics and Pragmatics 5(6). 1–69. Revised version of Roberts 1996. https://doi.org/10.3765/sp.5.6. ↗
  52. Schlangen, David. 2003. A coherence-based approach to the interpretation of non-sentential utterances in dialogue. Edinburgh: University of Edinburgh dissertation. https://doi.org/1842/318. ↗
  53. Stanley, Jason & Zoltan Szabó. 2000. On quantifier domain restriction. Mind and Language 15(2-3). 219–261. https://doi.org/10.1111/1468-0017.00130. ↗
  54. Stojnic, Una, Matthew Stone & Ernie Lepore. 2013. Deixis (even without pointing). Philosophical Perspectives 27(1). 502–525. https://doi.org/10.1111/ph pe.12033. ↗
  55. Thompson, Sandra A. & William C. Mann. 1987. Rhetorical structure theory: A framework for the analysis of texts. International Pragmatics Association Hunter, Asher, and Lascarides (IPrA) Papers in Pragmatics 1. 79–105. https://doi.org/10.1075/iprapip.1 .1.03tho. van Kuppevelt, Jan. 1996. Inferring from topics: Scalar implicatures as topic-dependent inferences. Linguistics and Philosophy 19(4). 393–443. https : //doi.org/10.1007/BF00630897. ↗
  56. Venant, Antoine & Nicholas Asher. 2016. Ok or not OK? Commitments, acknowledgments and corrections. Semantics and Linguistic Theory (SALT) 25. 595–614. https://doi.org/10.3765/salt.v25i0.3072. ↗
  57. Venant, Antoine, Nicholas Asher, Philippe Muller, Pascal Denis & Stergos Afantenos. 2013. Expressivity and comparison of models of discourse structure. In Maxine Eskenazi, Michael Strube, Barbara Di Eugenio & Jason D. Williams (eds.), Special interest group on discourse and dialogue (SIGdial 2013), 2–11. Metz. http://www.aclweb.org/anthology/W13-4002. ↗
  58. Wang, Li, Marco Lui, Su Nam Kim, Joakim Nivre & Timothy Baldwin. 2011. Predicting thread discourse structure over technical web forums. In Regina Barzilay & Mark Johnson (eds.), The conference on empirical methods in natural language processing (EMNLP 2011), 13–25. Edinburgh, United Kingdom: Association for Computational Linguistics. http://www.aclweb.org/anthology/D11-1002. ↗
  59. Webber, Bonnie. 1988. Tense as discourse anaphor. Computational Linguistics 14(2). 61–73. http://www.aclweb.org/anthology/J88-2006. ↗
  60. Yu, Haonan, Siddharth Narayanaswarmy, Andre Barbu & Jeff Siskind. 2015. A compositional framework for grounding language inference, generation, and acquisition in video. Journal of Artificial Intelligence Research (JAIR) 52. 601–713. https://doi.org/10.1613/jair.4556. ↗
  61. Zarrieß, Sina & David Schlangen. 2017. Obtaining referential word meanings from visual and distributional information: Experiments on object naming. The 55th Annual Meeting of the Association for Computational Linguistics (ACL) 1: Long papers. 243–254. https://doi.org/10.18653/v1/P17-1023. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/hunter-2018-formal-semantics-situated-conversation/