Elamur Research⌕
☰
home / papers / 2019 / Expectation and Locality Effects in the Prediction of Disfluent Fillers and Repairs in English Speech

Expectation and Locality Effects in the Prediction of Disfluent Fillers and Repairs in English Speech

Paper · 2019 · Samvit Dammalapati, Rajakrishnan Rajkumar and Sumeet Agarwal

Summary

The paper tests whether the places where speakers become disfluent can be accounted for by three psycholinguistic theories: Surprisal Theory, Dependency Locality Theory (DLT), and the hypothesis known as Uniform Information Density (UID). It uses the Switchboard corpus (via the Switchboard NXT release) and sorts words into three classes: reparandum, disfluent filler, and fluent word. A one-vs-all logistic regression is trained on features of the fluent words just before and just after each disfluency. Those features are trigram lexical surprisal, two UID variance measures, DLT integration and storage costs, and word duration. Each feature set is compared against a baseline that knows only speaker properties.

UID adds almost nothing. Surprisal helps modestly. Duration and DLT costs help a lot. The pattern suggests that disfluencies come right before hard material, and that speakers ease their load beforehand, partly by slowing down. Reparandums do not behave like fillers. The authors suggest this is because the reparandum words themselves may carry the easy, low-cost content. The work matters because it shows comprehension-difficulty metrics can say something about speech production.

Key points

  • Data and sampling: after filtering, the data held 14,923 reparandum cases, 12,183 filler cases and 558,361 fluent words. Each class was randomly downsampled to 12,183 for balance.
  • Baseline: the classifier uses only gender, age and speech rate, and reaches 37.18% accuracy. Every gain is tested against it with McNemar's test.
  • Lexical surprisal: computed from a trigram model trained on the Open American National Corpus with SRILM.
    • Adding surprisal on both sides gives 40.02%.
    • The following word alone adds about 1.6 points.
    • Words after disfluencies tend to have high surprisal.
  • UID and duration:
    • The best UID gain is only 0.42 points.
    • Duration of the preceding word gives a jump of 9.64 points, to 46.82%.
    • Surprisal and duration correlate at 0.49.
  • DLT costs: dependencies were obtained by stripping disfluencies from the constituency parses and converting them with the Stanford converter. DLT features gave the top accuracy of 48.57%, but integration cost effects pointed in unexpected directions.

Interesting ideas

  • I1. Instead of looking at the disfluent token itself, the authors take features from the fluent neighbours on each side, so fillers and repairs can be compared on equal terms.
  • I2. Integration cost (summed length of the dependencies closed at a word) and storage cost (count of still-open dependencies) are cheap per-word difficulty scores that fall straight out of a dependency tree.
  • I3. Each theory is judged by its accuracy gain over a baseline built only from nuisance variables, with McNemar's test for significance and regression weights to check the direction of the effect.
  • I4. Disfluencies are removed from constituency trees before converting them to dependency graphs, so the syntax is computed over the fluent skeleton.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tokens): when cleaning conversational input, Mova could flag likely filler or repair spots from cues on the surrounding words, rather than relying only on a fixed list of filler forms.
  • I2 (tree): these scores could be logged for every parsed word as an explainable "hard spot" signal, used to pick which sentences to route to stronger parsers or reviewers.
  • I3 (learn): new features or repair rules could be accepted only if they beat a nuisance-only control both significantly and in the predicted direction, which gives a clean negative control.
  • I4 (tree): a treebank built from speech transcripts could apply the same strip-then-convert step, keeping repair spans as side annotations instead of forcing them into the UD tree.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksPsychology · 1,649 worksExperimental and Cognitive Psychology · 930 works

🧭 Topics 2Research subjects and languages the work is about.English · 2,379 worksPsycholinguistics & language acquisition · 108 works

Cite

Cite as Samvit Dammalapati, Rajakrishnan Rajkumar and Sumeet Agarwal. 2019. Expectation and Locality Effects in the Prediction of Disfluent Fillers and Repairs in English Speech. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/n19-3015

@inproceedings{dammalapati2019expectation,
  title = {{Expectation and Locality Effects in the Prediction of Disfluent Fillers and Repairs in English Speech}},
  author = {Samvit Dammalapati, Rajakrishnan Rajkumar and Sumeet Agarwal},
  booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop},
  year = {2019},
  doi = {10.18653/v1/n19-3015},
  url = {https://doi.org/10.18653/v1/n19-3015}
}

References

31 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. and development. In [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing, volume 1, pages 517–520 vol.1. find
  2. Jennifer E Arnold, Maria Fagnano, and Michael K Tanenhaus. 2003. Disfluencies signal theee, um, new information. Journal of psycholinguistic research, 32(1):25–36. ↗
  3. John Hale. 2001. A probabilistic earley parser as a psycholinguistic model. In Proceedings of the second meeting of the North American Chapter of the Association for Computational Linguistics on Language technologies, pages 1–8. Association for Computational Linguistics.
  4. Jennifer E Arnold, Anthony Losongco, Thomas Wasow, and Ryan Ginstrom. 2000. Heaviness vs. newness: The effects of structural complexity and discourse status on constituent ordering. Language, 76(1):28–55. ↗
  5. Daphna Heller, Jennifer E Arnold, Natalie Klein, and Michael K Tanenhaus. 2015. Inferring difficulty: Flexibility in the real-time processing of disfluency. Language and speech, 58(2):190–203. ↗
  6. Dale J Barr. 2001. Trouble in mind: Paralinguistic indices of effort and uncertainty in communication. Oralité et gestualité: Interactions et comportements multimodaux dans la communication, pages 597– 600. find
  7. Matthew Honnibal and Mark Johnson. 2014. Joint incremental disfluency detection and dependency parsing. Transactions of the Association for Computational Linguistics, 2:131–142. ↗
  8. Alan Bell, Daniel Jurafsky, Eric Fosler-Lussier, Cynthia Girand, Michelle Gregory, and Daniel Gildea. 2003. Effects of disfluencies, predictability, and utterance position on word form variation in english conversation. The Journal of the Acoustical Society of America, 113(2):1001–1024. ↗
  9. Nancy Ide and Keith Suderman. 2004. The american national corpus first release. In LREC. find
  10. T Florian Jaeger and Roger P Levy. 2007. Speakers optimize information density through syntactic reduction. In Advances in neural information processing systems, pages 849–856. find
  11. Sasha Calhoun, Jean Carletta, Jason M Brenier, Neil Mayo, Dan Jurafsky, Mark Steedman, and David Beaver. 2010. The nxt-format switchboard corpus: a rich resource for investigating the syntax, semantics, pragmatics and prosody of dialogue. Language resources and evaluation, 44(4):387–419. ↗
  12. Ayush Jain, Vishal Singh, Sidharth Ranjan, Rajakrishnan Rajkumar, and Sumeet Agarwal. 2018. Uniform information density effects on syntactic choice in hindi. In Proceedings of the Workshop on Linguistic Complexity and Natural Language Processing, pages 38–48. find
  13. Herbert H Clark and Thomas Wasow. 1998. Repeating words in spontaneous speech. Cognitive psychology, 37(3):201–242. ↗
  14. Michael Xavier Collins. 2014. Information density and dependency length as complementary cognitive models. Journal of psycholinguistic research, 43(5):651–681. ↗
  15. Jeremy G Kahn, Matthew Lease, Eugene Charniak, Mark Johnson, and Mari Ostendorf. 2005. Effective use of prosody in parsing conversational speech. In Proceedings of the conference on human language technology and empirical methods in natural language processing, pages 233–240. Association for Computational Linguistics. Marie-Catherine De Marneffe, Bill MacCartney, Christopher D Manning, et al. Generating typed dependency parses from phrase structure parses.
  16. Roger Levy. 2008. Expectation-based syntactic comprehension. Cognition, 106(3):1126–1177. ↗
  17. Vera Demberg and Frank Keller. 2008. Data from eye-tracking corpora as evidence for theories of syntactic processing complexity. Cognition, 109(2):193–210. ↗
  18. Haitao Liu, Chunshan Xu, and Junying Liang. 2017. Dependency distance: A new perspective on syntactic patterns in natural languages. Physics of Life Reviews, 21:171 – 193. ↗
  19. Vera Demberg, Asad B. Sayeed, Philip J. Gorinski, and Nikolaos Engonopoulos. 2012. Syntactic surprisal affects spoken word duration in conversational contexts. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLP-CoNLL ’12, pages 356– 367, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  20. Gregory Scontras, William Badecker, Lisa Shank, Eunice Lim, and Evelina Fedorenko. 2015. Syntactic complexity effects in sentence production. Cognitive Science, 39(3):559–583. ↗
  21. Claude Elwood Shannon. 1948. A mathematical theory of communication. Bell system technical journal, 27(3):379–423. ↗
  22. Edward Gibson. 2000. The dependency locality theory: A distance-based theory of linguistic complexity. Image, language, brain, pages 95–126. ↗
  23. Elizabeth Shriberg, Rebecca Bates, and Andreas Stolcke. 1997. A prosody only decision-tree model for disfluency detection. In Fifth European Conference on Speech Communication and Technology. find
  24. J. J. Godfrey, E. C. Holliman, and J. McDaniel. 1992. Switchboard: telephone speech corpus for research Andreas Stolcke. 2002. Srilm-an extensible language modeling toolkit. In Seventh international conference on spoken language processing. find
  25. Trang Tran, Shubham Toshniwal, Mohit Bansal, Kevin Gimpel, Karen Livescu, and Mari Ostendorf. 2017. Parsing speech: a neural approach to integrating lexical and acoustic-prosodic information. arXiv preprint arXiv:1704.07287. ↗
  26. Jean E Fox Tree and Herbert H Clark. 1997. Pronouncing the as thee to signal problems in speaking. Cognition, 62(2):151–167. ↗
  27. Shravan Vasishth and Richard L. Lewis. 2006. Argument-head distance and processing complexity: Explaining both locality and antilocality effects. Language, 82(4):767–794. ↗
  28. Shaolei Wang, Wanxiang Che, Yue Zhang, Meishan Zhang, and Ting Liu. 2017. Transition-based disfluency detection using lstms. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2785–2794. find
  29. Vicky Zayats, Mari Ostendorf, and Hannaneh Hajishirzi. 2016. Disfluency detection using a bidirectional lstm. arXiv preprint arXiv:1604.03209. ↗
  30. Victoria Zayats, Mari Ostendorf, and Hannaneh Hajishirzi. 2014. Multi-domain disfluency and repair detection. In Fifteenth Annual Conference of the International Speech Communication Association. find
  31. Simon Zwarts and Mark Johnson. 2011. The impact of language models and loss functions on repair disfluency detection. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 703–711. Association for Computational Linguistics. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/dammalapati-2019-expectation-locality-effects-prediction-disfluent/