Elamur Research⌕
☰
home / papers / 2015 / Automatic Identification of Rhetorical Questions

Automatic Identification of Rhetorical Questions

Paper · 2015 · Shohini Bhattasali, Jeremy Cytryn, Elana Feldman et al.

Summary

The paper addresses rhetorical questions: utterances that look like ordinary questions but are used to make a claim rather than to request information. They are easy to confuse with genuine questions, yet telling them apart matters for summarization, QA, opinion mining and information extraction. The authors work with the Switchboard Dialog Act Corpus, keeping only question utterances together with the turn just before and just after each one. Each question is represented with word unigrams and bigrams plus POS bigrams and trigrams. Context features are kept in separate groups according to position (before or after) and to whether the same or a different speaker produced them. The n-grams are filtered by a class-predictivity score, and an SVM weighted for class imbalance is trained on them. Combining the question with both neighbouring turns gives the best F1, 53.71%, about ten points above a question-only baseline. This shows that dialogue context carries real signal for spotting non-literal questions.

Key points

  • Data: Switchboard Dialog Act Corpus, with 5960 training and 2555 test questions. Only about 5% of the test questions are rhetorical, so labelling everything non-rhetorical already scores 94.99% accuracy. For that reason precision, recall and F1 are reported.
  • Results by feature set:
    • Question only (baseline): F1 44.25%.
    • Question + Precedent + Subsequent: F1 53.71% (P 49.03, R 59.38, accuracy 94.87%).
    • Context alone, ignoring the question: F1 30.14%.
    • Following context was more informative than preceding context.
  • Classifiers: Naive Bayes and SVMLight (cost factor set from the class ratio) were tried, and the SVM was more robust as the feature space grew. POS n-grams came from the NLTK maxent treebank tagger. Highly predictive n-grams included "you know" and "well ,".
  • On a balanced 50/50 subset (670 train / 288 test), accuracy rose from 81.25% to 84.38% when context was added. McNemar's test did not reach significance there (p = 0.22) but did on the full skewed set (χ² = 30.74, p < 0.00001).
  • Fewer same-speaker context features were selected than different-speaker ones, which suggests the respondent's reaction is the stronger cue. Word n-grams were more useful than POS n-grams.

Interesting ideas

  • I1. Context features are split into four separate groups: before or after the question, crossed with same or different speaker. The same token can therefore mean different things depending on who said it and when, which suits any dialogue-level inference.
  • I2. The authors list known linguistic markers of rhetorical questions: strong NPIs such as "lift a finger", a modal combined with negation, and "after all" or "yet". They propose an NPI lexicon as future work, offering a cheap, interpretable signal beyond plain n-grams.
  • I3. Features are ranked by maximal predictivity, the highest maximum-likelihood P(class | n-gram) over any class, and the top j per group are kept, with j chosen by cross-validation. This is a simple, transparent way to surface the most discriminative cues.
  • I4. A context-only model, which never sees the question itself, still scores above chance. It works as a probe of how much signal lives outside the target utterance.
  • I5. On a heavily skewed task, the authors add a balanced subset and a McNemar test on paired predictions instead of trusting accuracy alone. They openly report where the gain is not yet significant.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (frames): The speech-act layer in prag could inspect the next turn's speaker and reaction (no answer, the asker keeps talking, a reply like "yeah") as explicit evidence when labelling a question as assertive rather than information-seeking.
  • I2 (frames): These markers could be matched as patterns over the UD tree (e.g., negation attached to a modal auxiliary, or an NPI idiom inside an interrogative clause), giving explainable triggers for a rhetorical-question speech act and its implied negative statement.
  • I3 (learn): The same score could rank candidate lexical or tree cues mined from silver data before they are turned into self-induced repair or classification rules, keeping the selection auditable.
  • I4 (learn): Evaluations could include such ablated "context-only" or "target-only" runs as controls, revealing whether a component relies on the target utterance or on its surroundings.
  • I5 (learn): Mova's evaluation suite could pair rare-phenomenon metrics (irony, rhetorical questions) with balanced test slices and paired significance tests, so small improvements are not overclaimed.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 9,086 worksArtificial Intelligence · 8,889 worksArts and Humanities · 9,296 worksLanguage and Linguistics · 9,206 works

Cite

Cite as Shohini Bhattasali, Jeremy Cytryn, Elana Feldman and Joonsuk Park. 2015. Automatic Identification of Rhetorical Questions. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.3115/v1/p15-2122

@inproceedings{bhattasali2015automatic,
  title = {{Automatic Identification of Rhetorical Questions}},
  author = {Shohini Bhattasali, Jeremy Cytryn, Elana Feldman and Joonsuk Park},
  booktitle = {Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)},
  year = {2015},
  doi = {10.3115/v1/p15-2122},
  url = {https://doi.org/10.3115/v1/p15-2122}
}

References

27 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jeremy Ang, Yang Liu, and Elizabeth Shriberg. 2005. Automatic dialog act segmentation and classification in multiparty meetings. In ICASSP (1), pages 1061–1064. find
  2. Rajesh Bhatt. 1998. Argument-adjunct asymmetries in rhetorical questions. In NELS 29. find
  3. Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural language processing with Python. O’Reilly Media, Inc. find
  4. Allen L. Edwards. 1948. Note on the “correction for continuity” in testing the significance of the difference between correlated proportions. In Psychometrika, 13(3):185–187. find
  5. Anastasia Giannakidou. 1999. Affective dependencies In Linguistics and Philosophy, 22(4): 367–421. find
  6. Springer John J. Godfrey, Edward C. Holliman, and Jane McDaniel. 1992. Switchboard: telephone speech corpus for research and development. In Acoustics, Speech, and Signal Processing, 1992. ICASSP-92., 1992 IEEE International Conference on, volume 1, pages 517–520 vol.1. find
  7. Sergio Grau, Emilio Sanchis, Marı́a José Castro, David Vilar. 2004. Dialogue act classification using a Bayesian approach In 9th Conference Speech and Computer. find
  8. Almuth Gresillon. 1980. Zum linguistischen Status rhetorischer Fragen InZeitschrift für germanistische Linguistik, 8(3): 273–289. find
  9. Chung-Hye Han. 1998. Deriving the interpretation of rhetorical questions. In Proceedings of West Coast Conference in Formal Linguistics, volume 16, pages 237–253. Citeseer. find
  10. T. Joachims. 1999. Making large-scale svm learning practical. Advances in kernel methods-support vector learning. ↗
  11. Dan Jurafsky, Rebecca Bates, Noah Coccaro, Rachel Martin, Marie Meteer, Klaus Ries, Elizabeth Shriberg, Andreas Stolcke, Paul Taylor, Carol V. Ess-Dykema, et al. 1997a. Automatic detection of discourse structure for speech recognition and understanding. In Automatic Speech Recognition and Understanding, 1997. Proceedings., 1997 IEEE Workshop on, pages 88–95. IEEE. ↗
  12. Dan Jurafsky, Elizabeth Shriberg, and Debra Biasca. 1997b. Switchboard SWBD-DAMSL shallow-discourse-function annotation coders manual. Technical Report Draft 13, University of Colorado, Institute of Cognitive Science. find
  13. Simon Keizer, Anton Nijholt, et al. 2002. Dialogue act recognition with bayesian networks for dutch dialogues. In Proceedings of the 3rd SIGdial workshop on Discourse and dialogue-Volume 2, pages 88–94. Association for Computational Linguistics.
  14. Katharina Morik, Peter Brockhausen, and T. Joachims. 1999. Combining statistical learning with a knowledge-based approach: a case study in intensive care monitoring. Technical report, Technical Report, SFB 475: Komplexitätsre-duktion in Multivariaten Datenstrukturen, Universität Dortmund. find
  15. Norbert Reithinger and Martin Klesen. 1997. Dialogue act classification using language models. In EuroSpeech. Citeseer. find
  16. Jerrold M. Saddock. 1971. Queclaratives In Seventh Regional Meeting of the Chicago Linguistic Society, 7: 223–232. find
  17. Jerrold M. Saddock. 1974. Toward a linguistic theory of speech acts Academic Press New York Ken Samuel, Sandra Carberry, and K. Vijay-Shanker. 1999. Automatically selecting useful phrases for dialogue act tagging. arXiv preprint cs/9906016. find
  18. Ken B. Samuel. 2000. Discourse learning: an investigation of dialogue act tagging using transformation-based learning. University of Delaware. find
  19. Elizabeth Shriberg, Andreas Stolcke, Dan Jurafsky, Noah Coccaro, Marie Meteer, Rebecca Bates, Paul Taylor, Klaus Ries, Rachel Martin, and Carol Van Ess-Dykema. 1998. Can prosody aid the automatic classification of dialog acts in conversational speech? Language and speech, 41(3-4):443–492. ↗
  20. Andreas Stolcke, Klaus Ries, Noah Coccaro, Elizabeth Shriberg, Rebecca Bates, Dan Jurafsky, Paul Taylor, Rachel Martin, Carol Van Ess-Dykema, and Marie Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational linguistics, 26(3):339–373. ↗
  21. Robert van Rooy. 2003. Negative polarity items in questions: Strength as relevance In Journal of Semantics, 20(3): 239–273. Oxford University Press. find
  22. Anand Venkataraman, Andreas Stolcke, and Elizabeth Shriberg. 2002. Automatic dialog act labeling with minimal supervision. In 9th Australian International Conference on Speech Science and Technology, SST 2002. find
  23. Daan Verbree, Rutger Rienks, and Dirk Heylen. 2006. Dialogue-act tagging using smart feature selection; results on multiple corpora. In Spoken Language Technology Workshop, 2006. IEEE, pages 70–73. IEEE. find
  24. Volker Warnke, Ralf Kompe, Heinrich Niemann, and Elmar Nöth. 1997. Integrated dialog act segmentation and classification using prosodic features and language models. In EU-ROSPEECH. find
  25. Nick Webb, Mark Hepple, and Yorik Wilks. 2005. Dialogue act classification based on intra-utterance features. In Proceedings of the AAAI Workshop on Spoken Language Understanding. Citeseer. find
  26. Klaus Zechner. 2002. Automatic summarization of open-domain multiparty dialogues in diverse genres. Computational Linguistics, 28(4):447– 485. ↗
  27. Matthias Zimmerman, Yang Liu, Elizabeth Shriberg, and Andreas Stolcke. 2005. A* based joint segentation and classification of dialog acts in multiparty meetings. In Automatic Speech Recognition and Understanding, 2005 IEEE Workshop on, pages 215–219. IEEE. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/bhattasali-2015-automatic-identification-rhetorical-questions/