Elamur Research⌕
☰
home / papers / 2024 / Filtered Corpus Training (FiCT) Shows that Language Models Can Generalize from Indirect Evidence

Filtered Corpus Training (FiCT) Shows that Language Models Can Generalize from Indirect Evidence

Paper · 2024 · Abhinav Patil, Jaap Jumelet, Yu Ying Chiu et al.

Summary

Do language models learn grammar by memorizing constructions they have seen, or can they infer rules for structures that never appear in their input? The authors propose Filtered Corpus Training (FiCT) to test this. They remove every sentence containing a target construction from a naturalistic training corpus, such as subjects modified by a prepositional phrase. They then train a fresh model from scratch on the reduced data and compare its grammaticality judgments with those of an identical model trained on the unfiltered corpus. Filters were built for many phenomena tested in BLiMP, including agreement, NPI licensing, quantifiers, binding, passives and determiner–noun agreement. Both LSTM and Transformer LMs with similar parameter counts were trained on each corpus. Transformers consistently achieved lower perplexity, yet both architectures generalized equally well. Withholding direct evidence usually cost only a small amount of accuracy, and scores stayed clearly above chance even on the benchmarks matching the removed constructions. This suggests LMs rely on indirect evidence rather than rote memory, and it shows that perplexity and linguistic generalization can diverge.

Key points

  • Evaluation uses BLiMP: 67 benchmarks of 1,000 minimal pairs each, covering twelve areas of morphology, syntax, semantics and their interface. A model succeeds on a pair when it assigns higher probability to the grammatical sentence.
  • Filtered corpora include AGR-PP-MOD, AGR-REL-CL, AGR-RE-IRR-SV, NPI-ONLY, NPI-SENT-NEG, NPI-SIM-QUES, QUANTIFIER-SUPERLATIVE, QUANTIFIER-EXISTENTIAL-THERE, BINDING-C-COMMAND, BINDING-CASE, BINDING-DOMAIN, BINDING-RECONSTRUCTION, PASSIVE, DET-ADJ-NOUN and DET-NOUN. These are compared against a FULL baseline.
  • Generalization is measured with acc Δ, the accuracy drop of the filtered model relative to the full-corpus model, and a related probability-based measure, P Δ. Low values indicate successful generalization from related constructions.
  • Generalization succeeded even where the source of indirect evidence is hard to identify, such as the superlative-quantifier filter. The authors suggest future work on explaining these mechanisms, citing earlier NPI and monotonicity analyses.
  • Code, data and links to all trained models on the HuggingFace Hub are available at github.com/CLMBRs/corpus-filtering. The paper appeared in TACL vol. 12 (2024), pp. 1597–1615.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🏢MicrosoftRedmond · United States378 works · 161,630 citations🎓University of WashingtonSeattle · United States402 works · 212,762 citations🎓University of AmsterdamAmsterdam · Netherlands151 works · 174,506 citations🏢AmazonSeattle · United States128 works · 41,697 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,894 works

Cite

Cite as Abhinav Patil, Jaap Jumelet, Yu Ying Chiu, Andy Lapastora, Peter Shen, Lexie Wang, Clevis Willrich and Shane Steinert-Threlkeld. 2024. Filtered Corpus Training (FiCT) Shows that Language Models Can Generalize from Indirect Evidence. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl_a_00720

@article{patil2024filtered,
  title = {{Filtered Corpus Training (FiCT) Shows that Language Models Can Generalize from Indirect Evidence}},
  author = {Abhinav Patil, Jaap Jumelet, Yu Ying Chiu, Andy Lapastora, Peter Shen, Lexie Wang, Clevis Willrich and Shane Steinert-Threlkeld},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2024},
  doi = {10.1162/tacl\_a\_00720},
  url = {https://doi.org/10.1162/tacl_a_00720}
}

References

51 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Liz Allen, Alison O’Connell, and Veronique Kiermer. 2019. How can we ensure visibility and diversity in research contributions? How the Contributor Role Taxonomy (CRediT) is helping the shift from authorship to contributorship. Learned Publishing, 32(1):71–74. https://doi.org/10.1002/leap.1210 — [doi:10.1002/leap.1210](https://doi.org/10.1002/leap.1210) ↗
  2. Suhas Arehalli, Brian Dillon, and Tal Linzen. 2022. Syntactic surprisal from neural models predicts, but underestimates, human processing difficulty from syntactic ambiguities. In Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL), pages 301–313, Abu Dhabi, United… find
  3. Emmanuel Chemla, Vincent Homer, and Daniel Rothschild. 2011. Modularity and intuitions in formal semantics: The case of polarity items. Linguistics and Philosophy, 34(6):537–570. https://doi.org/10.1007/s10988-012-9106-0 — [doi:10.1007/s10988-012-9106-0](https://doi.org/10.1007/s10988-012-9106-0) ↗
  4. Noam Chomsky. 1993. Lectures on Government and Binding. De Gruyter Mouton, Berlin, New York. https://doi.org/10.1515/9783110884166 ↗
  5. Milica Denić, Vincent Homer, Daniel Rothschild, and Emmanuel Chemla. 2021. The influence of polarity items on inferential judgments. Cognition, 215:104791. https://doi.org/10 .1016/j.cognition.2021.104791 find
  6. Allyson Ettinger. 2020. What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models. Transactions of the Association for Computational Linguistics, 8:34–48. https://doi.org/10 .1162/tacl_a_00298
  7. Victoria Fossum and Roger Levy. 2012. Sequential vs. hierarchical syntactic models of human incremental sentence processing. In Proceedings of the 3rd Workshop on Cognitive Modeling and Computational Linguistics (CMCL 2012), pages 61–69, Montréal, Canada. Association for Computational Linguistics. find
  8. Richard Futrell, Ethan Wilcox, Takashi Morita, Peng Qian, Miguel Ballesteros, and Roger Levy. 2019. Neural language models as psycholinguistic subjects: Representations of syntactic state. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Lingui…
  9. Jon Gauthier, Jennifer Hu, Ethan Wilcox, Peng Qian, and Roger Levy. 2020. SyntaxGym: An online platform for targeted evaluation of language models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 70–76, Online. Association for…
  10. Online. Association for Computational Linguistics. https://doi.org/10.18653/v1/2020.acl-main.158 ↗
  11. Adam Goodkind and Klinton Bicknell. 2018. Predictive power of word surprisal for reading times is a linear function of language model quality. In Proceedings of the 8th Workshop on Cognitive Modeling and Computational Linguistics (CMCL 2018), pages 10–18, Salt Lake City, Utah. Association for Comput…
  12. Dieuwke Hupkes, Mario Giulianelli, Verna Dankers, Mikel Artetxe, Yanai Elazar, Tiago Pimentel, Christos Christodoulopoulos, Karim Lasri, Naomi Saphra, Arabella Sinclair, Dennis Ulmer, Florian Schottmann, Khuyagbaatar Batsuren, Kaiser Sun, Koustuv Sinha, Leila Khalatbari, Maria Ryskina, Rita Frieske,… — Colorless Green Recurrent Networks Dream Hierarchically
  13. John Hale. 2001. A probabilistic Earley parser as a psycholinguistic model. In Second Meeting of the North American Chapter of the Association for Computational Linguistics. https://doi.org/10.3115/1073336 .1073357 ↗
  14. Yiding Hao, Simon Mendelsohn, Rachel Sterneck, Randi Martinez, and Robert Frank. 2020. Probabilistic predictions of people perusing: Evaluating metrics of language model performance for psycholinguistic modeling. In Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, pag… find
  15. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735, PubMed: 9377276 ↗
  16. Jennifer Hu, Jon Gauthier, Peng Qian, Ethan Wilcox, and Roger Levy. 2020. A systematic assessment of syntactic generalization in neural language models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1725–1744, Jaap Jumelet, Milica Denic, Jakub Szym… find
  17. Jaap Jumelet and Dieuwke Hupkes. 2018. Do language models understand anything? On the ability of LSTMS to understand negative polarity items. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 222–231, Brussels, Belgium. Association for C… find
  18. Katharina Kann, Alex Warstadt, Adina Williams, and Samuel R. Bowman. 2019. Verb argument structure alternations in word and sentence embeddings. In Proceedings of the Society for Computation in Linguistics (SCiL) 2019, pages 287–297. find
  19. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models. CoRR, abs/2001.08361.
  20. Najoung Kim and Tal Linzen. 2020. COGS: A compositional generalization challenge based on semantic interpretation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9087–9105, Online. Association for Computational Linguistics. https://doi.org /1… ↗
  21. Tatsuki Kuribayashi, Yohei Oseki, Takumi Ito, Ryo Yoshida, Masayuki Asahara, and Kentaro Inui. 2021. Lower perplexity is not always human-like. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language P… find
  22. Jey Han Lau, Alexander Clark, and Shalom Lappin. 2017. Grammaticality, acceptability, and probability: A probabilistic view of linguistic knowledge. Cognitive Science, 41(5):1202–1241. https://doi.org/10 .1111/cogs.12414 find
  23. Roger Levy. 2008. Expectation-based syntactic comprehension. Cognition, 106(3):1126–1177. https://doi.org/10.1016/j.cognition .2007.05.006 — [doi:10.1016/j.cognition](https://doi.org/10.1016/j.cognition) ↗
  24. Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016. Assessing the ability of LSTMs to learn syntax-sensitive dependencies. Transactions of the Association for Computational Linguistics, 4:521–535. https://doi.org/10.1162/tacl_a_00115 ↗
  25. Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. In International Conference on Learning Representations. find
  26. Rebecca Marvin and Tal Linzen. 2018. Targeted syntactic evaluation of language models. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1192–1202, Brussels, Belgium. Association for Computational Linguistics. https://doi.org/10.18653/v1/D18-1151 ↗
  27. Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3428–3448, Florence, Italy. Association for Computational Ling…
  28. Gary Milsark. 1974. Existential Sentences in English. Ph.D. thesis, MIT, Cambridge, MA. find
  29. Kanishka Misra and Kyle Mahowald. 2024. Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNs.
  30. Aaron Mueller and Tal Linzen. 2023. How to plant trees in language models: Data and architectural effects on the emergence of syntactic inductive biases. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11237–11252, Toronto, Ca… find
  31. Benjamin Newman, Kai-Siang Ang, Julia Gong, and John Hewitt. 2021. Refining targeted syntactic evaluation of language models. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3710–3723, Online. As… find
  32. Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Jan Hajič, Christopher D. Manning, Sampo Pyysalo, Sebastian Schuster, Francis Tyers, and Daniel Zeman. 2020. Universal Dependencies v2: An evergrowing multilingual treebank collection. In Proceedings of the Twelfth Language Resources and Eval…
  33. Joakim Nivre, Daniel Zeman, Filip Ginter, and Francis Tyers. 2017. Universal Dependencies. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Tutorial Abstracts. Valencia, Spain. Association for Computational Linguistics. find
  34. Byung-Doh Oh and William Schuler. 2023a. Transformer-based language model surprisal predicts human reading times best with about two billion training tokens. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1915–1921, Singapore. Association for Computational Linguistic… find
  35. Byung-Doh Oh and William Schuler. 2023b. Why does surprisal from larger transformer-based language models provide a poorer fit to human reading times? Transactions of the Association for Computational Linguistics, 11:336–350. https://doi.org/10.1162/tacl a 00548 ↗
  36. Byung-Doh Oh, Shisen Yue, and William Schuler. 2024. Frequency explains the inverse correlation of large language models’ size, training data amount, and surprisal’s fit to reading times. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (… find
  37. Judea Pearl. 2009. Causality: Models, Reasoning and Inference, 2nd edition. Cambridge University Press, USA. https://doi.org/10 .1017/CBO9780511803161 find
  38. Grusha Prasad, Marten van Schijndel, and Tal Linzen. 2019. Using priming to uncover the organization of syntactic representations in neural language models. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 66–76, Hong Kong, China. Association for Comput… find
  39. Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020. Stanza: A Python natural language processing toolkit for many human languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations.
  40. Kaiser Sun, Adina Williams, and Dieuwke Hupkes. 2023. The validity of evaluation results: Assessing concurrence across compositionality benchmarks. In Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL), pages 274–293, Singapore. Association for Computational Lingui… find
  41. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
  42. Alex Warstadt. 2022. Artificial Neural Networks as Models of Human Language Acquisition. Ph.D. thesis, New York University. https://doi.org/10.1201/9781003205388-2 — [doi:10.1201/9781003205388-2](https://doi.org/10.1201/9781003205388-2) ↗
  43. Alex Warstadt and Samuel R. Bowman. 2020. Can neural networks acquire a structural bias from raw linguistic data? In 42nd Annual Meeting of the Cognitive Science Society: Developing a Mind: Learning in Humans, Animals, and Machines, CogSci 2020, pages 1737–1743. find
  44. Alex Warstadt and Samuel R. Bowman. 2022. What artificial neural networks can tell us about human language acquisition. In Algebraic Structures in Natural Language, pages 17–60. CRC Press. https://doi.org/10.1201/9781003205388-2 ↗
  45. Alex Warstadt, Yu Cao, Ioana Grosu, Wei Peng, Hagen Blix, Yining Nie, Anna Alsop, Shikha Bordia, Haokun Liu, Alicia Parrish, Sheng-Fu Wang, Jason Phang, Anhad Mohananey, Phu Mon Htut, Paloma Jeretic, and Samuel R. Bowman. 2019a. Investigating BERT’s knowledge of language: Five analysis methods with…
  46. Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R. Bowman. 2020. BLiMP: The benchmark of linguistic minimal pairs for English. Transactions of the Association for Computational Linguistics, 8:377–392. https://doi.org/10.1162/tacl_a_00321 ↗
  47. Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. 2019b. Neural network acceptability judgments. Transactions of the Association for Computational Linguistics, 7:625–641. https://doi.org/10.1162/tacl a 00290 ↗
  48. Lucas Weber, Jaap Jumelet, Elia Bruni, and Dieuwke Hupkes. 2021. Language modelling as a multi-task problem. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 2049–2060, Online. Association for Computational Linguistics… find
  49. Lucas Weber, Jaap Jumelet, Elia Bruni, and Dieuwke Hupkes. 2024. Interpretability of language models via task spaces. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Bangkok, Thailand. Association for Computational Linguistics. http… find
  50. Jason Wei, Dan Garrette, Tal Linzen, and Ellie Pavlick. 2021. Frequency effects on syntactic rule learning in transformers. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 932–948, Online and Punta Cana, Dominican Republic. Association for Computation…
  51. Ethan Gotlieb Wilcox, Jon Gauthier, Jennifer Hu, Peng Qian, and Roger Philip Levy. 2020. On the predictive power of neural language models for human real-time comprehension behavior. In Proceedings of the Annual Meeting of the Cognitive Science Society, 42.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/patil-2024-filtered-corpus-training-fict-shows/