Elamur Research⌕
☰
home / papers / 2010 / Generating Tailored, Comparative Descriptions with Contextually Appropriate Intonation

Generating Tailored, Comparative Descriptions with Contextually Appropriate Intonation

Paper · 2010 · Michael White, Robert A. J. Clark and Johanna D. Moore

Summary

Spoken dialogue systems usually list every matching option in turn. In the DARPA Communicator evaluation, this presentation stage was the biggest contributor to dialogue length, and it also strains users' memory. The authors present FLIGHTS, a flight-information generator where the user's preferences guide every step of output production.

A multi-attribute decision model ranks the available flights. The system then mentions the top flight plus only those alternatives that offer a genuine trade-off, along with the attributes that matter for the decision. Because the presentation strategy is organised around these trade-offs, it directly yields both information structure (theme/rheme) and the content of referring expressions.

During surface realization in OpenCCG, Combinatory Categorial Grammar turns information structure into pitch accents and edge tones, and n-gram models decide phrase boundaries. The resulting prosodic markup steers a unit selection synthesizer.

Experts found the generated prosody significantly more acceptable than baseline n-gram predictions. Listeners rated the markup-driven voice as more natural than two baseline voices. Overall, the work shows that connecting content planning to intonation improves spoken, preference-tailored comparisons.

Key points

  • FLIGHTS integrates four components in a single pipeline: decision-theoretic user models, strategic content planning, prosody-aware CCG realization (OpenCCG) and unit selection synthesis.
  • The approach is illustrated with three hypothetical user models (Student, Frequent Flyer, Business Class) choosing flights from Edinburgh to Brussels. Each model yields different selected flights, referring expressions, aggregation, scalar terms and discourse cues such as "but" and "though".
  • The mapping from themes/rhemes to intonational phrases is one-to-many. Rules from the grammar are combined with data-driven n-gram choices about the type and placement of edge tones.
  • Evaluation had three parts: an expert prosody assessment (with ToBI annotations supplied by an expert), a listener perception test against two baseline unit selection voices, and an f0 analysis of theme phrases. All three favoured the generator-driven intonation.
  • One stated limitation concerns general-purpose voices: a Festival APML voice would need markup for the 2,000–3,000 out-of-domain utterances in its core database. The authors suggest a simplified annotation scheme as a way forward.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Speech & spoken language · 480 works

Cite

Cite as Michael White, Robert A. J. Clark and Johanna D. Moore. 2010. Generating Tailored, Comparative Descriptions with Contextually Appropriate Intonation. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/coli.2010.09-023-r1-08-002

@article{white2010generating,
  title = {{Generating Tailored, Comparative Descriptions with Contextually Appropriate Intonation}},
  author = {Michael White, Robert A. J. Clark and Johanna D. Moore},
  journal = {Computational Linguistics},
  year = {2010},
  doi = {10.1162/coli.2010.09-023-r1-08-002},
  url = {https://doi.org/10.1162/coli.2010.09-023-r1-08-002}
}

References

80 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aylett, Matthew. 2005. Merging data driven and rule based prosodic models for unit selection TTS. In 5th ISCA Speech Synthesis Workshop, pages 55–59, Pittsburg, PA. find
  2. Baker, Rachel Elizabeth. 2003. Using unit selection to synthesise contextually appropriate intonation in limited domain synthesis. Master’s thesis, Department of Linguistics, University of Edinburgh. find
  3. Baldridge, Jason and Geert-Jan Kruijff. 2002. Coupling CCG and Hybrid Logic Dependency Semantics. In Proceedings of 40th Annual Meeting of the Association for Computational Linguistics, pages 319–326, Philadelphia, PA. find
  4. Bangalore, Srinivas and Owen Rambow. 2000. Exploiting a probabilistic hierarchical model for generation. In Proceedings of COLING-00, pages 42–48, Saarbrucken, Germany.
  5. Bilmes, Jeff and Katrin Kirchhoff. 2003. Factored language models and general parallelized backoff. In Proceedings of HLT-03, pages 4–6, Edmonton, Canada. find
  6. Black, Alan W. and Keiichi Tokuda. 2005. The blizzard challenge—2005: Evaluating corpus-based speech synthesis on common datasets. In Interspeech 2005, pages 77–80, Lisbon. find
  7. Blackburn, Patrick. 2000. Representation, reasoning, and relational structures: A hybrid logic manifesto. Logic Journal of the IGPL, 8(3):339–625. find
  8. Bos, Johan, Ewan Klein, Oliver Lemon, and Tetsushi Oka. 2003. DIPPER: Description and formalisation of an information-state update dialogue system architecture. In 4th SIGdial Workshop on Discourse and Dialogue, pages 115–124, Sapporo. find
  9. Brenier, Jason, Ani Nenkova, Anubha Kothari, Laura Whitton, David Beaver, and Dan Jurafsky. 2006. The (non)utility of linguistic features for predicting prominence in spontaneous speech. IEEE/ACL 2006 Workshop on Spoken Language Technology, pages 54–57, Palm Beach, Aruba. find
  10. Bulyko, Ivan and Mari Ostendorf. 2002. Efficient integrated response generation from multiple targets using weighted finite state transducers. Computer Speech and Language, 16:533–550. find
  11. Calhoun, Sasha, Malvina Nissim, Mark Steedman, and Jason Brenier. 2005. A framework for annotating information structure in discourse. Proceedings of the ACL-05 Workshop on Frontiers in Corpus Annotation II: Pie in the Sky, pages 45–52, Ann Arbor, Michigan.
  12. Carberry, Sandra, Jennifer Chu-Carroll, and Stephanie Elzer. 1999. Constructing and utilizing a model of user preferences in collaborative consultation dialogues. Computational Intelligence Journal, 15(3):185–217. find
  13. Carenini, Giuseppe and Johanna D. Moore. 2000. A strategy for generating evaluative arguments. In Proceedings of INLG-00, pages 47–54, Mitzpe Ramon. find
  14. Carenini, Giuseppe and Johanna D. Moore. 2001. An empirical study of the influence of user tailoring on evaluative argument effectiveness. In Proceedings of IJCAI-01, pages 1307–1314, Seattle, WA. find
  15. Carenini, Giuseppe and Johanna D. Moore. 2006. Generating and evaluating evaluative arguments. Artificial Intelligence, 170:925–952. find
  16. Carroll, John, Ann Copestake, Dan Flickinger, and Victor Poznański. 1999. An efficient chart generator for (semi-) lexicalist grammars. In Proceedings of EWNLG-99, pages 86–95, Toulouse, France. find
  17. Carroll, John and Stefan Oepen. 2005. High efficiency realization for a wide-coverage unification grammar. In Proceedings of IJCNLP-05, pages 165–176, Jeju Island, Korea. find
  18. Chung, Grace. 2004. Developing a flexible spoken dialog system using simulation. In Proceedings of ACL ’04, pages 63–70, Barcelona. find
  19. Clark, Robert A. J. and Simon King. 2006. Joint prosodic and segmental unit selection speech synthesis. In Proceedings of Interspeech 2006, pages 1312–1315, Pittsburgh, PA. find
  20. Clark, Robert A. J., Monika Podsiadlo, Mark Fraser, Catherine Mayo, and Simon King. 2007. Statistical analysis of the Blizzard Challenge 2007 listening test results. In Proceedings of Blizzard Workshop (in Proceedings of the 6th ISCA Workshop on Speech Synthesis), pages 1–6, Bonn, Germany. find
  21. Clark, Robert A. J., Korin Richmond, and Simon King. 2007. Multisyn: Open-domain unit selection for the Festival speech synthesis system. Speech Communication, 49(4):317–330. find
  22. Currie, K. and A. Tate. 1991. O-Plan: The open planning architecture. Artificial Intelligence, 52:49–86. de Carolis, Bernadina, Catherine Pelachaud, Isabella Poggi, and Mark Steedman. 2004. APML, a Mark-up Language for Believable Behavior Generation. In H. Prendinger and M. Ishizuka, editors, Life-l… find
  23. Demberg, Vera and Johanna D. Moore. 2006. Information presentation in spoken dialogue systems. In Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics (EACL-06), pages 5–72, Trento, Italy. find
  24. Dusterhoff, Kurt E., Alan W. Black, and Paul A. Taylor. 1999. Using decision trees within the tilt intonation model to predict f0 contours. In Eurospeech-99, pages 1627–1630, Budapest, Hungary. find
  25. Edwards, W. and F. H. Barron. 1994. SMARTS and SMARTER: Improved simple methods for multiattribute utility measurement. Organizational Behavior and Human Decision Processes, 60:306–325. find
  26. Elhadad, Michael. 1993. Using Argumentation to Control Lexical Choice: A Functional Unification Implementation. Ph.D. thesis, Columbia University. find
  27. Foster, Mary Ellen and Michael White. 2004. Techniques for Text Planning with XSLT. In Proceedings of the 4th NLPXML Workshop, pages 1–8, Barcelona, Spain. find
  28. Fraser, Mark and Simon King. 2007. find
  29. The Blizzard Challenge 2007. In Proceedings of Blizzard Workshop (in Proceedings of the 6th ISCA Workshop on Speech Synthesis), pages 7–12, Bonn, Germany. find
  30. Ginzburg, Jonathan. 1996. Interrogatives: Questions, facts and dialogue. In Shalom Lappin, editor, The Handbook of Contemporary Semantic Theory. Blackwell, Oxford, pages 385–422. find
  31. Hirschberg, Julia. 1993. Pitch accent in context: Predicting intonational prominence from text. Artificial Intelligence, 63:305–340. find
  32. Hirschberg, Julia and Pilar Prieto. 1996. Training intonational phrasing rules automatically for English and Spanish text-to-speech. Speech Communication, 18:281–290. find
  33. Hitzeman, Janet, Alan W. Black, Chris Mellish, Jon Oberlander, Massimo Poesio, and Paul Taylor. 1999. An annotation scheme for concept-to-speech synthesis. In Proceedings of EWNLG-99, pages 59–66, Toulouse. find
  34. Hitzeman, Janet, Alan W. Black, Chris Mellish, Jon Oberlander, and Paul Taylor. 1998. On the use of automatically generated discourse-level information in a concept-to-speech synthesis system. In Proceedings of ICSLP-98, pages 2763–2768, Sydney, Australia. find
  35. Kay, Martin. 1996. Chart generation. In Proceedings of ACL-96, pages 200–204, Santa Cruz, USA.
  36. Knight, Kevin and Vasileios Hatzivassiloglou. 1995. Two-level, many-paths generation. In Proceedings of ACL-95, pages 252–260, Cambridge, MA. find
  37. Kruijff, Geert-Jan M. 2003. Binding across boundaries. In Geert-Jan M. Kruijff and Richard T. Oehrle, editors, Resource-Sensitivity in Binding and Anaphora. Kluwer Academic Publishers, pages 123–158, Dordrecht, The Netherlands. find
  38. Kruijff-Korbayová, Ivana, Stina Ericsson, Kepa J. Rodrı́guez, and Elena Karagjosova. 2003. Producing contextually appropriate intonation in an information-state based dialogue system. In Proceedings of EACL-93, pages 227–234, Budapest, Hungary. find
  39. Langkilde, Irene. 2000. Forest-based statistical sentence generation. In Proceedings of NAACL-00, pages 170–177, Seattle, Washington. find
  40. Langkilde-Geary, Irene. 2002. An empirical verification of coverage and correctness for a general-purpose sentence generator. In Proceedings of INLG-02, pages 17–24, New York, NY.
  41. Linden, Greg, Steve Hanks, and Neal Lesh. 1997. Interactive assessment of user preference models: The automated travel assistant. In Proceedings of User Modeling ’97, pages 67–78, Chia Laguna, Sardinia, Italy. find
  42. Marsi, Erwin. 2004. Optionality in evaluating prosody prediction. In Proceedings of the 5th ISCA Speech Synthesis Workshop, pages 13–18, Pittsburg, PA. find
  43. Martin, D. L., A. J. Cheyer, and D. B. Moran. 1999. The Open Agent Architecture: A framework for building distributed software systems. Applied Artificial Intelligence, 13(1):91–128. find
  44. Mayo, C., R. A. J. Clark, and S. King. 2005. Multidimensional scaling of listener responses to synthetic speech. In Interspeech 2005, pages 1725–1728, Lisbon. find
  45. Moore, Johanna, Mary Ellen Foster, Oliver Lemon, and Michael White. 2004. Generating tailored, comparative descriptions in spoken dialogue. In Proceedings of FLAIRS-04, pages 917–922, Miami Beach, USA. find
  46. Moore, Robert C. 2002. A complete, efficient sentence-realization algorithm for unification grammar. In Proceedings of INLG-02, pages 41–48, New York, NY. find
  47. Nakatsu, Crystal and Michael White. 2006. Learning to say it well: Reranking realizations by predicted synthesis quality. In Proceedings of COLING-ACL ’06, pages 1113–1120, Sydney, Australia. find
  48. Oh, Alice H. and Alexander I. Rudnicky. 2002. Stochastic natural language generation for spoken dialog systems. Computer, Speech & Language, 16(3/4):387–407. find
  49. Pan, Shimei, Kathleen McKeown, and Julia Hirschberg. 2002. Exploring features from natural language generation for prosody modeling. Computer Speech and Language, 16:457–490. find
  50. Papineni, Kishore, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2001. Bleu: A method for Automatic Evaluation of Machine Translation. Technical Report RC22176, IBM.
  51. Pierrehumbert, Janet. 1980. The Phonology and Phonetics of English Intonation. Ph.D. thesis, MIT. find
  52. Polifroni, Joseph, Grace Chung, and Stephanie Seneff. 2003. Towards automatic generation of mixed-initiative dialogue systems from Web content. In Proceedings of Eurospeech ’03, pages 193–196, Geneva. find
  53. Pon-Barry, Heather, Fuliang Weng, and Sebastian Varges. 2006. Evaluation of content presentation strategies for an in-car spoken dialogue system. In Proceedings of Interspeech 2006, pages 1930–1933, Pittsburgh, PA. find
  54. Prevost, Scott. 1995. A Semantics of Contrast and Information Structure for Specifying Intonation in Spoken Language Generation. Ph.D. thesis, University of Pennsylvania. find
  55. Ratnaparkhi, Adwait. 2002. Trainable approaches to surface natural language generation and their application to conversational dialog systems. Computer, Speech & Language, 16(3/4):435–455. find
  56. Reiter, Ehud and Robert Dale. 2000. Building Natural Language Generation Systems. Cambridge University Press, Cambridge. find
  57. Ristad, Eric S.˙1995. A Natural Law of Succession. Technical Report CS-TR-495-95, Princeton University. find
  58. Roberts, Craige. 1996. Information structure: Towards an integrated formal theory of pragmatics. Ohio State University Working Papers in Linguistics, 49:91–136. find
  59. Rocha, Neide Franca. 2004. Evaluating prosodic markup in a spoken dialogue system. Master’s thesis, Department of Linguistics, University of Edinburgh. find
  60. Rooth, Mats. 1992. A theory of focus interpretation. Natural Language Semantics, 1:75–116. find
  61. Shemtov, Hadar. 1997. Ambiguity Management in Natural Language Generation. Ph.D. thesis, Stanford University. find
  62. Silverman, K., M. Beckman, J. Pitrelli, M. Ostendorf, C. Wightman, P. Price, J. Pierrehumbert, and J. Hirschberg. 1992. ToBI: A standard for labeling English prosody. Proceedings of ICSLP92, 2:867–870, Banff, Canada. find
  63. Snover, Matthew, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of the Association for Machine Translation in the Americas (AMTA-06), pages 223–231, Cambridge, MA.
  64. Steedman, Mark. 2000a. Information structure and the syntax-phonology interface. Linguistic Inquiry, 31(4):649–689. find
  65. Steedman, Mark. 2000b. The Syntactic Process. MIT Press, Cambridge, MA. find
  66. Steedman, Mark. 2004. Using APML to specify intonation. Magicster Project Deliverable 2.5. University of Edinburgh. Available at http://www.ltg.ed.ac.uk/magicster/deliverables/annex2.5/apml-howto.pdf ↗
  67. Steedman, Mark. 2006. Information-structural semantics for English intonation. In Chungmin Lee, Matt Gordon, and Daniel Büring, editors, Topic and Focus: Cross-Linguistic Perspectives on Meaning and Intonation. Springer, Dordrecht, pages 245–264. find
  68. Stolcke, Andreas. 2002. SRILM — An extensible language modeling toolkit. In Proceedings of ICSLP-02, pages 901–904, Denver, Colorado. find
  69. Taylor, Paul and Alan Black. 1998. Assigning phrase breaks from part-of-speech sequences. Computer Speech and Language, 12:99–117. find
  70. Taylor, P., A. Black, and R. Caley. 1998. The architecture of the the Festival speech synthesis system. In Third International Workshop on Speech Synthesis, pages 147–151, Sydney. find
  71. Theune, Mariët. 2002. Contrast in concept-to-speech generation. Computer Speech and Language, 16:491–531. find
  72. Theune, Mariët, Esther Klabbers, Jan-Roelof de Pijper, Emiel Krahmer, and Jan Odijk. 2001. From data to speech: A general approach. Natural Language Engineering, 7(1):47–86. find
  73. Vallduvı́, Enric and Maria Vilkuna. 1998. On rheme and kontrast. In Peter Culicover and Louise McNally, editors, Syntax and Semantics, Vol. 29: The Limits of Syntax. Academic Press, San Diego, CA, pages 79–108. van Deemter, Kees, Emiel Krahmer, and Mariët Theune. 2005. Real versus template-based na… find
  74. Walker, M. A., S. Whittaker, A. Stent, P. Maloor, J. D. Moore, M. Johnston, and G. Vasireddy. 2002. Speech-plans: Generating evaluative responses in spoken dialogue. In Proceedings of INLG ’02, pages 73–80, New York, NY. find
  75. Walker, M. A., S. J. Whittaker, A. Stent, P. Maloor, J. D. Moore, M. Johnston, and G. Vasireddy. 2004. Generation and evaluation of user-tailored responses in multimodal dialogue. Cognitive Science, 28:811–840. find
  76. Walker, Marilyn A., Rebecca Passonneau, and Julie E. Boland. 2001. Quantitative and qualitative evaluation of DARPA Communicator spoken dialogue systems. In Proceedings of ACL-01, pages 515–522, Toulouse, France. find
  77. Walker, Marilyn A., Owen C. Rambow, and Monica Rogati. 2002. Training a sentence planner for spoken dialogue using boosting. Computer Speech and Language, 16:409–433. find
  78. White, Michael. 2004. Reining in CCG Chart Realization. In Proceedings of INLG-04, pages 182–191, Brockenhurst, UK. find
  79. White, Michael. 2006a. CCG chart realization from disjunctive inputs. In Proceedings of INLG-06, pages 12–19, Sydney, Australia. find
  80. White, Michael. 2006b. Efficient realization of coordinate structures in Combinatory Categorial Grammar. Research on Language & Computation, 4(1):39–75. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/white-2010-generating-tailored-comparative-descriptions-contextually/