Elamur Research⌕
☰
home / papers / 2025 / Automatic Extraction of Clausal Embedding Based on Large-Scale English Text Data

Automatic Extraction of Clausal Embedding Based on Large-Scale English Text Data

Paper · 2025 · Iona Carslaw, Sivan Milton, Nicolas Navarre et al.

Summary

Linguistic work on clausal complements, such as what wonder or be happy can embed, has mostly relied on invented example sentences judged by native speakers. Such data say little about how these constructions are actually used. The authors build a pipeline that finds and labels English embedded clauses in raw web-scale text. It uses constituency parses from the Berkeley Neural Parser (via spaCy) plus hand-written heuristics. For each hit, the tool returns three things: the embedding predicate's tokens, the clause span, and the clause type (declarative, polar, alternative or constituent interrogative).

To evaluate it, they built GECS, a hand-annotated benchmark of naturally occurring sentences drawn from Dolma. GECS also includes adversarial lookalikes such as relative clauses, free relatives and unconditionals. They then ran the tool over a Dolma v1.6 sample and released the resulting dataset. A case study on emotive factives shows the corpus statistics largely support the textbook generalisation while still surfacing a few plausible counter-examples. This makes the resource useful for testing syntax–semantics theories against real usage.

Key points

  • GECS contents: 147 declarative, 138 polar interrogative, 84 alternative interrogative and 158 constituent interrogative embedded clauses, plus 111 adversarial sentences with no embedded clause.
    • Annotation was done by one researcher and checked by a second.
  • Candidate pool: 866,538 uncleaned sentences from Dolma files (cc_en_head-0000, cc_en_head-0001, c4-0085).
    • Sentences were prefiltered by SBAR presence, then by whether/if or wh-words.
  • Prior resource: MegaAcceptability slotted 1007 verbs into 50 templates, giving about 50,000 crowd-rated sentences on a 7-point scale.
    • It covers only a few complementisers and prepositions and excludes adjectival and complex predicates.
  • Emotive factive case study: predicates such as happy, surprised and afraid embedded 175,479 declarative and 47,877 constituent interrogative clauses, against only 159 polar and 134 alternative ones.
    • Most of those polar and alternative hits were false positives.
  • Release: code is at github.com/navarrenicolas/clause_parser, and the extracted dataset is on HuggingFace (nnavarre/Embedded_Clauses-dolma_v1_6-sample).
    • The authors rejected LLM-based extraction because a black box prevents thorough error analysis.

Interesting ideas

  • I1. Pair the positive gold set with a curated set of hard negatives, i.e. structures that look like the target but are not (relatives, free relatives, unconditionals), so precision is measured where it actually breaks.
  • I2. Encode the embedding predicate as a token list that skips negation and adjuncts, kept separate from the clause span and the clause type. This yields a clean lexical key for counting predicate–complement pairings.
  • I3. Use cheap surface filters (a structural marker, then lexical cues like whether/if or wh-words) to shrink a huge raw pool before costly human or model annotation.
  • I4. Check a theoretical selectional generalisation against corpus counts, then sort the rare violations into error types (unconditional, wrong predicate span, non-factive reading, adjunct) before treating any of them as genuine exceptions.
  • I5. Offer a recursive representation that makes coordinated and nested embedded clauses explicit, rather than flat spans.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): add negative-control suites of lookalike constructions to evaluation, so a parser or repair rule that over-labels ccomp is caught.
  • I2 (frames): normalise multiword predicates like be happy about into such keys when building frames, so clausal arguments attach to the right predicate lexeme.
  • I3 (learn): prefilter silver-data candidates by surface cues so LLM-teacher or witness-parser budget goes to sentences that actually contain the phenomenon.
  • I4 (world): derive verb/adjective complement-selection tables from parsed corpora, and use this error taxonomy to clean them before compiling them into verb-class knowledge.
  • I5 (tree): represent nested and coordinated ccomp structures (including ellipsis under coordination) recursively, so downstream frames can follow which proposition sits inside which.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.English · 2,263 works

Cite

Cite as Iona Carslaw, Sivan Milton, Nicolas Navarre, Ciyang Qing and Wataru Uegaki. 2025. Automatic Extraction of Clausal Embedding Based on Large-Scale English Text Data. Proceedings of the Society for Computation in Linguistics 2025.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{carslaw2025automatic,
  title = {{Automatic Extraction of Clausal Embedding Based on Large-Scale English Text Data}},
  author = {Iona Carslaw, Sivan Milton, Nicolas Navarre, Ciyang Qing and Wataru Uegaki},
  booktitle = {Proceedings of the Society for Computation in Linguistics 2025},
  year = {2025},
  url = {https://aclanthology.org/2025.scil-1.27/}
}

References

26 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Klaus Abels. 2004. Why surprise-predicates do not embed polar interrogatives. Linguistische Arbeitsberichte, 81:203–222. find
  2. Ivano Caponigro. 2003. Free not to ask: On the semantics of free relatives and wh-words cross-linguistically. Ph.D. thesis, University of California, Los Angeles. find
  3. Wayne Cowart. 1997. Experimental Syntax: Applying Objective Methods to Sentence Judgments, 1 edition. SAGE Publications. find
  4. Jon Robert Gajewski. 2007. Neg-raising and polarity. Linguistics and Philosophy, 30:289–328. ↗
  5. Jane Grimshaw. 1979. Complement selection and the lexicon. Linguistic inquiry, 10(2):279–326. find
  6. Matthew Honnibal and Ines Montani. 2017. spaCy 2: Natural language understanding with Bloom embeddings, convolutional neural networks and incremental parsing. To appear. find
  7. Larry Horn. 1978. Remarks on neg-raising. Syntax and Semantics, 9. ↗
  8. Lauri Karttunen. 1977. Syntax and semantics of questions. Linguistics and philosophy, 1:3–44. ↗
  9. Nikita Kitaev, Steven Cao, and Dan Klein. 2019. Multilingual constituency parsing with self-attention and pre-training. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3499–3505, Florence, Italy. Association for Computational Linguistics. ↗
  10. Nikita Kitaev and Dan Klein. 2018. Constituency parsing with a self-attentive encoder. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2676–2686, Melbourne, Australia. Association for Computational Linguistics. ↗
  11. Clemens Mayr. 2019. Triviality and interrogative embedding: context sensitivity, factivity, and neg-raising. Natural language semantics, 27(3):227–278. 330 9 ↗
  12. Petr Pajas, Jan Štěpánek, and Michal Sedlák. 2009. PML tree query. LINDAT/CLARIAH-CZ digital library at the Institute of Formal and Applied Linguistics (ÚFAL), Faculty of Mathematics and Physics, Charles University. find
  13. Kyle Rawlins. 2008. (Un)conditionals: An investigation in the syntax and semantics of conditional structures. Ph.D. thesis, University of California, Santa Cruz. find
  14. Phillip Resnik and Aaron Elkiss. 2005. The linguist’s search engine: An overview. find
  15. Kjell Johan Sæbø. 2007. A whether forecast. In Logic, Language, and Computation: 6th International Tbilisi Symposium on Logic, Language, and Computation, TbiLLC 2005 Batumi, Georgia, September 12-16, 2005. Revised Selected Papers 6, pages 189–199. find
  16. Carson T. Schütze. 2016. The empirical base of linguistics : grammaticality judgments and linguistic methodology. Language Science Press, Berlin. ↗
  17. Micah Shlain, Hillel Taub-Tabib, Shoval Sadde, and Yoav Goldberg. 2020. Syntactic search by example. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 17–23. find
  18. Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani, Oyvind Tafjord, Evan Pete Walsh, Hannaneh Hajishirzi, Noah A. Smith, Luke Zettlemoyer, Iz Beltagy, Dirk Groeneveld, Jesse Dodge, and Kyle Lo. 2023. Dolma: An Open Corpus of Three Trillion Tokens for Language Model Pretraining Research. arXiv preprint. ↗
  19. Nadine Theiler, Floris Roelofsen, and Maria Aloni. 2019. Picky predicates: why believe doesn’t like interrogative complements, and other puzzles. Natural language semantics, 27(2):95–134. ↗
  20. Wataru Uegaki and Yasutada Sudo. 2019. The* hope-wh puzzle. Natural Language Semantics, 27(4):323– 356. ↗
  21. Henk van Riemsdijk. 2006. Free relatives. In Martin Everaert and Henk van Riemsdijk, editors, The Blackwell companion to syntax, volume 2, pages 338–382. Blackwell, Oxford. find
  22. Aaron Steven White. 2021. On believing and hoping whether. Semantics and pragmatics, 14(6):1–21. ↗
  23. Aaron Steven White and Kyle Rawlins. 2016. A computational model of s-selection. Proceedings from Semantics and Linguistic Theory, 26(26):641–663. ↗
  24. Aaron Steven White and Kyle Rawlins. 2020. Frequency, acceptability, and selection: A case study of clause-embedding. Glossa (London), 5(1):1–40. ↗
  25. Hedde Zeijlstra. 2018. Does neg-raising involve neg-raising? Topoi, 37(3):417–433. ↗
  26. Richard Zuber. 1982. Semantic restrictions on certain complementizers. In Proceedings of the 13th International Congress of Linguists, pages 434–436. Tokyo. *Figure 5: False-positive parse of a sentence with a conditional* *Figure 4: The data structure in GECS for coordination and nesting of embedded clauses.* find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/carslaw-2025-automatic-extraction-clausal-embedding-based/