Elamur Research⌕
☰
home / papers / 2024 / That's Optional: A Contemporary Exploration of “that” Omission in English Subordinate Clauses

That's Optional: A Contemporary Exploration of “that” Omission in English Subordinate Clauses

Paper · 2024 · Ella Rabinovich

Summary

The paper asks whether the Uniform Information Density (UID) hypothesis explains when writers drop the optional complementizer "that" before a subordinate clause, as in "she said [that] he left". Earlier computational work used about 6,700 spoken SwitchBoard sentences and a bigram model. This study builds a corpus of roughly 100K informal written sentences from Reddit. Clause structure was found with the benepar parser and rule filters, and further processing used spacy. Each sentence is described by classic predictors: main clause and subordinate clause length, main verb frequency, and distance to the SC subject. Two information measures computed with a pretrained LLM are added. The first is surprisal of the subordinate clause's first word given the whole main clause, marginalized over whether "that" is present. The second, which is new, is the entropy of the model's next-word distribution at that point. A logistic regression shows that surprisal and entropy are the strongest predictors: hard-to-predict or highly uncertain continuations favour keeping "that". The released data and code support further study of syntactic reduction.

Key points

  • Source data: 2M Reddit posts and comments (2020–2022), from over 20K users across more than 5K threads.
  • Filtering: sentences were limited to 5–50 words, leaving 487,614.
  • Extracted classes: 40,786 sentences with explicit "that" SCONJ, 57,845 with an omitted one, and 51,802 with other uses of "that". Released at github.com/ellarabi/uid-that-sc-omission.
  • Manual check: 500 sampled sentences were reviewed. The reviewer judged whether adding or removing "that" kept the sentence valid and its meaning unchanged. 96.4% of explicit items and 95.7% of implicit items passed.
  • Verb coverage: 434 distinct main verb lemmas occur. The top-10 (e.g. "think", "say", "know") cover 64.7% of cases. Omission rates differ strongly by verb: "think" and "guess" mostly drop "that", while others are balanced. The lemma alone is therefore not a sufficient cue.
  • Regression results (scaled coefficients):
    • All verbs: SC onset entropy has the largest coefficient (≈0.43) and surprisal ≈0.30. Main clause length is also positive and significant.
    • "think" only: surprisal leads (≈0.46), and SC subject distance becomes insignificant.
  • Surprisal vs. frequency: surprisal correlates with onset frequency (Pearson r −0.57 and −0.47). Swapping surprisal for frequency gives a somewhat weaker model.

Interesting ideas

  • I1. Implicit cases are found by applying the same structural rules as for explicit "that" but requiring the word to be missing, which gives matched positive and negative samples of an optional element.
  • I2. Instead of retraining a language model on text with "that" stripped out, the onset probability is combined with and without "that" inserted, removing a circularity at no training cost.
  • I3. Entropy of the whole next-word distribution is used alongside the surprisal of the actual word, and the two carry complementary signal about how uncertain a point in the sentence is.
  • I4. Before analysis, a small sample of extracted sentences was checked by hand with a meaning-preservation test (does adding or removing the element keep the sentence valid?).
  • I5. Testing robustness by refitting the same model on a single frequent verb ("think") separates lexical effects from general ones.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): The same rule pairing could check that the parser builds the same ccomp attachment with and without a mark, catching cases where an omitted complementizer derails the analysis.
  • I2 (learn): When an LLM teacher scores silver data, optional function words could be marginalized the same way so that scores do not just reward their presence.
  • I3 (back): When generating text from the meaning graph, the system could insert "that" (or other optional markers) where the upcoming clause start is surprising or the continuation highly uncertain, making output read more naturally.
  • I4 (learn): Self-induced repair rules could be validated by sampling their outputs and applying such a reversible-edit check, with failure rates reported like a negative control.
  • I5 (frames): Per-verb slices could be used when evaluating speech-act or reporting-verb frames, so that findings are not driven by one dominant lemma.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Academic College of Tel Aviv-YafoTel Aviv · Israel2 works · 5 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Arts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksNeuroscience · 186 worksCognitive Neuroscience · 185 works

🧭 Topics 1Research subjects and languages the work is about.English · 2,379 works

Cite

Cite as Ella Rabinovich. 2024. That's Optional: A Contemporary Exploration of “that” Omission in English Subordinate Clauses. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2024.acl-short.36

@inproceedings{rabinovich2024that,
  title = {{That's Optional: A Contemporary Exploration of “that” Omission in English Subordinate Clauses}},
  author = {Ella Rabinovich},
  booktitle = {Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2024},
  doi = {10.18653/v1/2024.acl-short.36},
  url = {https://doi.org/10.18653/v1/2024.acl-short.36}
}

References

23 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Wasim Ahmed, Peter A Bath, and Gianluca Demartini. 2017. Using twitter as a data source: An overview of ethical, legal, and methodological challenges. The ethics of online research, 2:79–107. find
  2. Ben Ambridge and Adele E Goldberg. 2008. The island status of clausal complements: Evidence in favor of an information structure explanation. find
  3. Matthew Aylett and Alice Turk. 2006. Language redundancy predicts syllabic duration and the spectral characteristics of vocalic syllable nuclei. The Journal of the Acoustical Society of America, 119(5). find
  4. Alan Bell, Daniel Jurafsky, Eric Fosler-Lussier, Cynthia Girand, Michelle Gregory, and Daniel Gildea. 2003. Effects of disfluencies, predictability, and utterance position on word form variation in english conversation. The Journal of the acoustical society of America, 113(2):1001–1024. find
  5. Adrian Benton, Glen Coppersmith, and Mark Dredze. 2017. Ethical research protocols for social media health research. In Proceedings of the first ACL workshop on ethics in natural language processing, pages 94–102.
  6. Thomas Hikaru Clark, Clara Meister, Tiago Pimentel, Michael Hahn, Ryan Cotterell, Richard Futrell, and Roger Levy. 2023. A Cross-Linguistic Pressure for Uniform Information Density in Word Order. Transactions of the Association for Computational Linguistics, 11:1048–1065.
  7. Michael Xavier Collins. 2014. Information density and dependency length as complementary cognitive models. Journal of Psycholinguistic Research, 43. find
  8. Nomi Erteschik-Shir. 1997. The dynamics of focus structure. Cambridge University Press. find
  9. Austin F Frank and T Florain Jaeger. 2008. Speaking rationally: Uniform information density as an optimal strategy for language production. In Proceedings of the annual meeting of the cognitive science society, volume 30. find
  10. Dmitriy Genzel and Eugene Charniak. 2002. Entropy rate constancy in text. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 199–206.
  11. John A Hawkins. 2004. Efficiency and complexity in grammars. OUP Oxford. find
  12. T Florian Jaeger. 2010. Redundancy and reduction: Speakers manage syntactic information density. Cognitive psychology, 61(1):23–62. ↗
  13. Glorianna Jagfeld, Fiona Lobban, Paul Rayson, and Steven H Jones. 2021. Understanding who uses reddit: Profiling individuals with a self-reported bipolar disorder diagnosis. arXiv preprint arXiv:2104.11612. ↗
  14. Roger Levy and T Jaeger. 2006. Speakers optimize information density through syntactic reduction. Advances in neural information processing systems, 19. find
  15. Kyle Mahowald, Evelina Fedorenko, Steven T Piantadosi, and Edward Gibson. 2013. Info/information theory: Speakers choose shorter words in predictive contexts. Cognition, 126(2):313–318. find
  16. Luke Maurits, Dan Navarro, and Amy Perfors. 2010. Why are some word orders more common than others? a uniform information density account. Advances in neural information processing systems, 23. find
  17. Clara Meister, Tiago Pimentel, Patrick Haller, Lena Jäger, Ryan Cotterell, and Roger Levy. 2021. Revisiting the uniform information density hypothesis. arXiv preprint arXiv:2109.11635. ↗
  18. Maeve Olohan and Mona Baker. 2000. Reporting that in translated english. evidence for subconscious processes of explicitation? Across languages and cultures, 1(2):141–158. find
  19. Tiago Pimentel, Clara Meister, Ethan G Wilcox, Roger P Levy, and Ryan Cotterell. 2023. On the effect of anticipation on reading times. Transactions of the Association for Computational Linguistics, 11:1624– 1642. find
  20. Nicholas Proferes, Naiyan Jones, Sarah Gilbert, Casey Fiesler, and Michael Zimmer. 2021. Studying reddit: A systematic overview of disciplines, approaches, methods, and ethics. Social Media+ Society, 7(2):20563051211019004. find
  21. Ella Rabinovich, Masih Sultani, and Suzanne Stevenson. 2019. Codeswitch-reddit: Exploration of written multilingual discourse in online discussion forums. arXiv preprint arXiv:1908.11841. ↗
  22. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30.
  23. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. 2022. Opt: Open pre-trained transformer language models.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/rabinovich-2024-that-s-optional-contemporary-exploration/