Elamur Research⌕
☰
home / papers / 2026 / Can Language Models Learn Typologically Implausible Languages?

Can Language Models Learn Typologically Implausible Languages?

Paper · 2026 · Tianyang Xu, Tatsuki Kuribayashi, Yohei Oseki et al.

Summary

Languages around the world show strong word-order correlations. For example, verb–object languages usually have prepositions, and object–verb languages usually have postpositions. These patterns are often explained by human learning biases, but it is debated whether such biases are specific to language or arise from general cognition. Xu et al. use language models as controllable learners to test whether a system with no built-in linguistic constraints still prefers typologically plausible grammars.

The authors start from real English (head-initial) and Japanese (head-final) corpora. From these, they build naturalistic counterfactual versions, each of which flips a single head–dependent ordering drawn from the Greenberg/Dryer correlation pairs. They refined these corpora through repeated annotation and corner-case correction. They then trained autoregressive and masked LMs from scratch and measured learnability through per-token perplexity, minimal-pair preferences, and the BLiMP and JBLiMP benchmarks.

The models learned the subtly implausible languages more slowly, although on some metrics they eventually caught up. This suggests that harmonic word-order preferences can emerge from domain-general learning. It also challenges the view that LMs acquire attested and unattested languages with equal ease.

Key points

  • The counterfactual corpora alter one grammatical property at a time and are built symmetrically for English and Japanese. This targets the boundary between plausible and implausible languages more finely than Kallini et al. (2024), whose test languages were "impossible" ones.
  • The manipulated orderings come from Dryer's (1992) correlation pairs, which are based on a sample of 625 languages and extend Greenberg's (1963) survey of 30 languages.
  • Learnability was assessed with two LM types (autoregressive and masked) using three measures: perplexity, minimal pairs contrasting original and swapped word orders, and BLiMP/JBLiMP accuracy.
  • Final perplexity mostly showed no significant difference between original and counterfactual languages. The <V, O> variants were the noisiest and had the highest perplexity.
  • The authors flag a key limitation: parser errors introduce noise into the counterfactual data, which may partly explain the effects. Code and models are released on GitHub (sally-xu-42).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of California San DiegoSan Diego · United States73 works · 51,948 citations🎓The University of TokyoTokyo · Japan90 works · 9,156 citations🎓Toyota Technological Institute at ChicagoChicago · United States33 works · 13,261 citations🎓ETH ZurichZurich · Switzerland72 works · 5,176 citations🎓Mohamed bin Zayed University of Artificial IntelligenceAbu Dhabi · United Arab Emirates124 works · 5,676 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksDevelopmental and Educational Psychology · 499 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Tianyang Xu, Tatsuki Kuribayashi, Yohei Oseki, Ryan Cotterell and Alex Warstadt. 2026. Can Language Models Learn Typologically Implausible Languages?. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1162/tacl.a.640

@article{xu2026can,
  title = {{Can Language Models Learn Typologically Implausible Languages?}},
  author = {Tianyang Xu, Tatsuki Kuribayashi, Yohei Oseki, Ryan Cotterell and Alex Warstadt},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.1162/tacl.a.640},
  url = {https://doi.org/10.1162/tacl.a.640}
}

References

91 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mark Baker. 2015. Case: Its Principles and Parameters. Cambridge University Press. Number: 146. find
  2. Noam Chomsky. 1965. Aspects of the Theory of Syntax. MIT Press. find
  3. Mark C. Baker. 2009. The macroparameter in a microparametric world. In The Limits of Syntactic Variation, pages 351–373. find
  4. Noam Chomsky. 1981. Lectures on Government and Binding. https://doi.org/10 .1515/9783110884166 find
  5. Marco Baroni. 2022. On the proper role of linguistically-oriented deep net analysis in linguistic theorizing. In Shalom Lappin and Jean Philippe Bernardy, editors, Algebraic Structures in Natural Language, pages 1–16. CRC Press. find
  6. Noam Chomsky and Howard Lasnik. 1993. The theory of principles and parameters. In Syntax: An International Handbook of Contemporary Research. Walter de Gruyter. find
  7. Noam Chomsky, Ian Roberts, and Jeffrey Watumull. 2023. Noam Chomsky: The false promise of ChatGPT. The New York Times. find
  8. Jon Barwise and Robin Cooper. 1988. Generalized quantifiers and natural language. In Jack Kulas, James H. Fetzer, and Terry L. Rankin, editors, Philosophy, Language, and Artificial intelligence: Resources for Processing Natural Language, pages 241–301. Springer Netherlands, Dordrecht. https://doi.or… — [doi:10.1007/978-94-009-2727-8_10](https://doi.org/10.1007/978-94-009-2727-8_10) ↗
  9. Guglielmo Cinque. 2017. A microparametric approach to the head-initial/head-final parameter. Linguistic Analysis, 41(3–4):309–366. find
  10. Alexander Clark and Shalom Lappin. 2011. Linguistic Nativism and the Poverty of the Stimulus. John Wiley & Sons. find
  11. Thomas Hikaru Clark, Clara Meister, Tiago Pimentel, Michael Hahn, Ryan Cotterell, Richard Futrell, and Roger Levy. 2023. A cross-linguistic pressure for uniform information density in word order. Transactions of the Association for Computational Linguistics, 11:1048–1065. https://doi.org/10 .1162/ta…
  12. Grégoire Delétang, Anian Ruoss, Jordi Grau-Moya, Tim Genewein, Li Kevin Wenliang, Elliot Catt, Chris Cundy, Marcus Hutter, Shane Legg, Joel Veness, and Pedro A. Ortega. 2023. Neural networks and the chomsky hierarchy. In the Eleventh International Conference on Learning Representations.
  13. Ionut Constantinescu, Tiago Pimentel, Ryan Cotterell, and Alex Warstadt. 2024. Investigating critical period effects in language acquisition through neural language models. https://doi.org/10.1162/tacl a 00725 ↗
  14. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An image is worth 16x16 words: Transformers for image recognition at scale. In Intern…
  15. Ailı́s Cournane. 2017. In defence of the child in-novator. In Eric Mathieu and Robert Truswell, editors, Micro-change and macro-change in diachronic syntax, pages 10–36. Oxford University Press. find
  16. Matthew Dryer. 1992. The Greenbergian word order correlations. Language, 68:138–181. find
  17. Jennifer Culbertson, Julie Franck, Guillaume Braquet, Magda Barrera Navarro, and Inbal Arnon. 2020. A learning bias for word order harmony: Evidence from speakers of nonharmonic languages. Cognition, 204:104392. https://doi.org/10.1016/j.cognition .2020.104392, PubMed: 32673786 — [doi:10.1016/j.cognition](https://doi.org/10.1016/j.cognition) ↗
  18. Brian DuSell and David Chiang. 2022. Learning hierarchical structures with differentiable nondeterministic stacks. In International Conference on Learning Representations. find
  19. Javid Ebrahimi, Dhruv Gelda, and Wei Zhang. 2020. How can self-attention networks recognize Dyck-n languages? In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4301–4306, Online. Association for Computational Linguistics. https://doi.org/10.18653/v1 /2020.findings-emnlp…
  20. Jennifer Culbertson and Simon Kirby. 2016. Simplicity and specificity in language: Domain-general biases have domain-specific effects. Frontiers in Psychology, 6. https://doi.org/10.3389/fpsyg.2015.01964 — [doi:10.3389/fpsyg.2015.01964](https://doi.org/10.3389/fpsyg.2015.01964) ↗
  21. Jennifer Culbertson and Simon Kirby. 2022. Syntactic harmony arises from a domain-general learning bias. In Proceedings of the 44th Annual Conference of the Cognitive Science Society. Number: 44. find
  22. Maryia Fedzechkina, T. Florian Jaeger, and Elissa L. Newport. 2012. Language learners restructure their input to facilitate efficient communication. Proceedings of the National Academy of Sciences, 109(44):17897–17902. https://doi.org/10.1073/pnas .1215776109, PubMed: 23071337 — [doi:10.1073/pnas](https://doi.org/10.1073/pnas) ↗
  23. Jennifer Culbertson and Elissa L. Newport. 2015. Harmonic biases in child learners: In support of language universals. Cognition, 139:71–82. https://doi.org/10.1016/j.cognition .2015.02.007 — [doi:10.1016/j.cognition](https://doi.org/10.1016/j.cognition) ↗
  24. Jennifer Culbertson, Paul Smolensky, and Géraldine Legendre. 2012. Learning biases predict a word order universal. Cognition, 122(3):306–329. https://doi.org/10.1016/j.cognition.2011.10.017, PubMed: 22208785 ↗
  25. Riki Fujihara, Tatsuki Kuribayashi, Kaori Abe, Ryoko Tokuhisa, and Kentaro Inui. 2022. Topicalization in language models: A case study on Japanese. In Proceedings of the 29th International Conference on Computational Linguistics, pages 851–862, Gyeongju, Republic of Korea, International Committee on… find
  26. Marie-Catherine De Marneffe, Christopher D. Manning, Joakim Nivre, and Daniel Zeman. find
  27. Joseph H. Greenberg. 1963. Some universals of grammar with particular reference to the order of meaningful elements. Universals of language, 2:73–113. find
  28. United Arab Emirates (Hybrid), Association for Computational Linguistics. https://doi.org/10.18653/v1/2022.conll-1.7 Mandy Guo, Zihang Dai, Denny Vrandečić, and Rami Al-Rfou. 2020. Wiki-40B: Multilingual language model dataset. In Proceedings of the Twelfth Language Resources and Evaluation Confe… ↗
  29. Anne S. Hsu, Nick Chater, and Paul Vitányi. 2013. Language learning from positive evidence, reconsidered: A simplicity-based approach. Topics in Cognitive Science, 5(1):35–55. https://doi.org/10.1111/tops.12005, PubMed: 23335573 ↗
  30. Michael Hahn, Dan Jurafsky, and Richard Futrell. 2020. Universals of word order reflect optimization of grammars for efficient communication. Proceedings of the National Academy of Sciences, 117(5):2347–2353. https://doi.org/10.1073/pnas.1910923117, PubMed: 31964811 — [doi:10.1073/pnas.1910923117](https://doi.org/10.1073/pnas.1910923117) ↗
  31. John T. Hale and Miloš Stanojević. 2024. Do LLMs learn a true syntactic universal? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17106–17119, Miami, Florida, USA. Association for Computational Linguistics. https://doi.org/10 .18653/v1/2024.emnlp-m… find
  32. Jennifer Hu, Jon Gauthier, Peng Qian, Ethan Wilcox, and Roger Levy. 2020. A systematic assessment of syntactic generalization in neural language models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1725–1744, Online. Association for Computational… find
  33. Tim Hunter. 2025. Kallini et al. (2024) do not compare impossible languages with constituency-based ones. Computational Linguistics, 51:641–650. https://doi.org/10.1162/coli_a_00554 ↗
  34. Yiding Hao, Dana Angluin, and Robert Frank. 2022. Formal language recognition by hard attention transformers: Perspectives from circuit complexity. Transactions of the Association for Computational Linguistics, 10:800–810. https://doi.org/10.1162/tacl a 00490 ↗
  35. Marc D. Hauser, Noam Chomsky, and W. Tecumseh Fitch. 2002. The faculty of language: What is it, who has it, and how did it evolve? Science, 298(5598):1569–1579. https://doi.org/10.1126/science .298.5598.1569, PubMed: 12446899 — [doi:10.1126/science](https://doi.org/10.1126/science) ↗
  36. John A. Hawkins. 1983. Word Order Universals. Academic Press, San Diego. https://doi.org/10.1016/B978-0-12-333370-4 .50002-4 — [doi:10.1016/b978-0-12-333370-4](https://doi.org/10.1016/b978-0-12-333370-4) ↗
  37. Larry M. Hyman. 2008. Universals in phonology. The Linguistic Review, 25(1–2):83–137. https://doi.org/10.1515/TLIR.2008 .003 — [doi:10.1515/tlir.2008](https://doi.org/10.1515/tlir.2008) ↗
  38. Jaap Jumelet, Milica Denic, Jakub Szymanik, Dieuwke Hupkes, and Shane Steinert-Threlkeld. 2021. Language models use monotonicity to assess NPI licensing. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4958–4969, Online. Association for Computational Linguistics.… find
  39. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. DeBERTa: Decoding enhanced BERT with disentangled attention. In International Conference on Learning Representations.
  40. Jaap Jumelet and Dieuwke Hupkes. 2018. Do language models understand anything? On the ability of LSTMs to understand negative polarity items. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 222–231, Brussels, Belgium. Association for C… find
  41. Mark Hopkins. 2022. Towards more natural artificial languages. In Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL), pages 85–94, Abu Dhabi, John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ…
  42. Jordan Kodner, Sarah Payne, and Jeffrey Heinz. 2023. Why linguistics will thrive in the 21st century: A reply to Piantadosi (2023). arXiv preprint arXiv:2308.03228. — [arXiv:2308.03228](https://arxiv.org/abs/2308.03228) find
  43. Takumitsu Kudo. 2005. Mecab: Yet another part-of-speech and morphological analyzer. find
  44. Tatsuki Kuribayashi, Ryo Ueda, Ryo Yoshida, Yohei Oseki, Ted Briscoe, and Timothy Baldwin. 2024. Emergent word order universals from cognitively-motivated language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14522–… find
  45. Julie Kallini, Isabel Papadimitriou, Richard Futrell, Kyle Mahowald, and Christopher Potts. 2024. Mission: Impossible language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14691–14714, Bangkok, Thailand. Association…
  46. Nur Lan, Emmanuel Chemla, and Roni Katzir. 2024. Large language models and the argument from the poverty of the stimulus. Linguistic Inquiry, pages 1–28. https://doi.org/10 .1162/ling_a_00533 find
  47. Tal Linzen. 2019. What can linguistics and deep learning contribute to each other? Response to Pater. Language, 95(1):e99–e108. https://doi.org/10.1353/lan.2019.0015 — [doi:10.1353/lan.2019.0015](https://doi.org/10.1353/lan.2019.0015) ↗
  48. Carla L. Hudson Kam and Elissa L. Newport. 2005. Regularizing unpredictable variation: The roles of adult and child learners in language formation and change. Language Learning and Development, 1(2):151–195. https://doi.org/10.1080/15475441.2005.9684215 — [doi:10.1080/15475441.2005.9684215](https://doi.org/10.1080/15475441.2005.9684215) ↗
  49. Rebecca Marvin and Tal Linzen. 2018. Targeted syntactic evaluation of language models. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1192–1202, Brussels, Belgium. Association for Computational Linguistics. https://doi.org/10.18653/v1/D18-1151 ↗
  50. Roni Katzir. 2023. Why large language models are poor theories of human linguistic cognition: A reply to Piantadosi. Biolinguistics, 17(e13153). https://doi.org/10.5964/bioling.13153 ↗
  51. Sam Whitman McGrath, Jacob Russin, Ellie Pavlick, and Roman Feiman. 2024. How can deep neural networks inform theory in psychological science? Current Directions in Psychological Science, 33(5):325–333. https://doi.org/10.1177/09637214241268098, PubMed: 39949337 — [doi:10.1177/09637214241268098](https://doi.org/10.1177/09637214241268098) ↗
  52. Richard S. Kayne. 1994. The Antisymmetry of Syntax, volume 25, MIT Press. find
  53. Richard S. Kayne. 2013. Why are there no directionality parameters? Studies in Chinese Linguistics, 34(1). find
  54. Raphäl Milliére. 2024. Language models as models of language. In R. Nefdt, G. Dupre, and K. Stanton, editors, The Oxford Handbook of the Philosophy of Linguistics. Oxford University Press.
  55. Simon Kirby, Hannah Cornish, and Kenny Smith. 2008. Cumulative cultural evolution in the laboratory: An experimental approach to the origins of structure in human language. Proceedings of the National Academy of Sciences, 105(31):10681–10686. https://doi.org/10.1073/pnas.0707835105, PubMed: 1866769… — [doi:10.1073/pnas.0707835105](https://doi.org/10.1073/pnas.0707835105) ↗
  56. Kanishka Misra and Kyle Mahowald. 2024. Language models learn rare phenomena from less rare phenomena: The case of the missing AANNs. In Proceedings of the 2024 conference on empirical methods in natural language processing, pages 913–929, Miami, Florida, USA. Association for Computational Linguisti…
  57. Joe Pater. 2019. Generative linguistics and neural networks at 60: Foundation, friction, and fusion. Language, 95(1):e41–e74. https://doi.org/10.1353/lan.2019.0009 — [doi:10.1353/lan.2019.0009](https://doi.org/10.1353/lan.2019.0009) ↗
  58. Abhinav Patil, Jaap Jumelet, Yu Ying Chiu, Andy Lapastora, Peter Shen, Lexie Wang, Clevis Willrich, and Shane Steinert-Threlkeld. 2024. Filtered corpus training (FiCT) shows that language models can generalize from indirect evidence. Transactions of the Association for Computational Linguistics, 12:…
  59. Jeff Mitchell and Jeffrey Bowers. 2020. Pri-orless recurrent networks learn curiously. In Proceedings of the 28th International Conference on Computational Linguistics, pages 5147–5158, Barcelona, Spain (Online). International Committee on Computational Linguistics. https://doi.org/10 .18653/v1/2020… find
  60. Andrea Moro, Matteo Greco, and Stefano F. Cappa. 2023. Large languages, impossible languages and human brains. Cortex, 167:82–85. https://doi.org/10.1016/j.cortex .2023.07.003, PubMed: 37540953 — [doi:10.1016/j.cortex](https://doi.org/10.1016/j.cortex) ↗
  61. Yugo Murawaki. 2019. On the definition of Japanese word. find
  62. Andy Perfors, Joshua B. Tenenbaum, and Terry Regier. 2011. The learnability of abstract syntactic principles. Cognition, 118(3):306–338. https://doi.org/10.1016/j.cognition .2010.11.001, PubMed: 21186021 — [doi:10.1016/j.cognition](https://doi.org/10.1016/j.cognition) ↗
  63. Jackson Petty and Robert Frank. 2021. Transformers generalize linearly. arXiv:2109.12036 [cs]. ArXiv: 2109.12036. — [arXiv:2109.12036](https://arxiv.org/abs/2109.12036) find
  64. Alain Peyraube. 1912. L’évolution des formes grammaticales. Scientia; rivista di scienza, 6(12):384. https://doi.org/10.3406/lgge.2002.2400 ↗
  65. Mariacristina Musso, Andrea Moro, Volkmar Glauche, Michel Rijntjes, Jürgen Reichenbach, Christian Büchel, and Cornelius Weiller. 2003. Broca’s area and the language instinct. Nature Neuroscience, 6(7):774–781. Publisher: Nature Publishing Group US New York. https://doi.org/10.1038/nn1077, PubMed:… — [doi:10.1038/nn1077](https://doi.org/10.1038/nn1077) ↗
  66. Steven T. Piantadosi. 2023, Modern language models refute Chomsky’s approach to language. In Edward Gibson and Moshe Poliak, editors, From Fieldwork to Linguistic Theory: A Tribute to Dan Everett, volume 15, Empirically Oriented Theoretical Morphology and Syntax, pages 353–414. Language Science Pres… find
  67. Frederick J. Newmeyer. 2005. Possible and Probable Languages: A Generative Perspective on Linguistic Typology. Oxford University Press. find
  68. Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Jan Hajič, Christopher D. Manning, Sampo Pyysalo, Sebastian Schuster, Francis Tyers, and Daniel Zeman. 2020. Universal Dependencies v2: An evergrowing multilingual treebank collection. In Proceedings of the Twelfth Language Resources and Eval…
  69. Gregory Pringle. 2016. Thoughts on the Universal Dependencies proposal for Japanese: The problem of the word as a linguistic unit. find
  70. Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020. Stanza: A Python natural language processing toolkit for many human languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 101–108, Online.…
  71. Hisashi Noda. 1996. Wa to ga (Wa and ga). Kurosio Publishers. find
  72. Mai Omura, Aya Wakasa, and Masayuki Asahara. 2021. Word delimitation issues in UD Japanese. In Proceedings of the Fifth Workshop on Universal Dependencies (UDW, SyntaxFest 2021), pages 142–150, Sofia, Bulgaria. Association for Computational Linguistics. find
  73. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. find
  74. Shauli Ravfogel, Yoav Goldberg, and Tal Linzen. 2019. Studying the inductive biases of RNNs with synthetic variations of natural languages. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Lon…
  75. Vienna, Austria. Association for Computational Linguistics. https://doi.org/10 .18653/v1/2025.acl-long.1357 find
  76. Taiga Someya, Ryo Yoshida, and Yohei Oseki. 2024. Targeted syntactic evaluation on the Chomsky hierarchy. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 15595–15605, Torino, Italia. ELRA and ICCL. ht…
  77. Edward P. Stabler. 2013. Two models of minimalist, incremental syntactic analysis. Topics in Cognitive Science, 5(3):611–633. https://doi.org/10.1111/tops.12031, PubMed: 23757195 — [doi:10.1111/tops.12031](https://doi.org/10.1111/tops.12031) ↗
  78. David Samuel, Andrey Kutuzov, Lilja Øvrelid, and Erik Velldal. 2023. Trained on 100 million words and still in shape: BERT meets British National Corpus. In Findings of the Association for Computational Linguistics: EACL 2023, pages 1954–1974, Dubrovnik, Croatia. Association for Computational Lingui… find
  79. Lena Strobl, William Merrill, Gail Weiss, David Chiang, and Dana Angluin. 2024. What formal languages can transformers express? A survey. Transactions of the Association for Computational Linguistics, 12:543–561. https://doi.org/10.1162/tacl a 00663 ↗
  80. Takaaki Tanaka, Yusuke Miyao, Masayuki Asahara, Sumire Uematsu, Hiroshi Kanayama, Shinsuke Mori, and Yuji Matsumoto. 2016. Universal Dependencies for Japanese. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 1651–1658, Portorož, Slovenia. E… find
  81. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725, Berlin, Germany. Association for Computational Lin…
  82. Kazuhiro Teruya. 2004. Metafunctional profile of the grammar of Japanese. Language typology. A functional Perspective, pages 185–254. https://doi.org/10.1075/cilt.253 .06ter Taiga Someya and Yohei Oseki. 2023. JBLiMP: Japanese benchmark of linguistic minimal pairs. In Findings of the Association for… ↗
  83. Kazuhiro Teruya. 2007. A Systemic Functional Grammar of Japanese. Bloomsbury Academic. find
  84. Natsuko Tsujimura. 2013. An Introduction to Japanese Linguistics. John Wiley & Sons. find
  85. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
  86. Taiga Someya, Anej Svete, Brian DuSell, Timothy J. O’Donnell, Mario Giulianelli, and Ryan Cotterell. 2025. Information locality as an inductive bias for neural language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2…
  87. Science, 30(5):945–982. https://doi.org/10.1207/s15516709cog0000 89, PubMed: 21702842 — [doi:10.1207/s15516709cog0000](https://doi.org/10.1207/s15516709cog0000) ↗
  88. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoe… — Transformers: State-of-the-Art Natural Language Processing
  89. Alex Warstadt. 2022. Artificial Neural Networks as Models of Human Language Acquisition. PhD Thesis, New York University. find
  90. Alex Warstadt and Samuel R. Bowman. 2022. What artificial neural networks can tell us about human language acquisition. In Shalom Lappin and Jean-Philippe Bernardy, editors, Algebraic Structures in Natural Language, pages 17–60. CRC Press.
  91. Alex Warstadt, Aaron Mueller, Leshem Choshen, Ethan Wilcox, Chengxu Zhuang, Juan Ciro, Rafael Mosquera, Bhargavi Paranjabe, Adina Williams, Tal Linzen, and Ryan Cotterell. 2023. Findings of the BabyLM Challenge: Sample-efficient pretraining on developmentally plausible corpora. In Proceedings of the… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/xu-2026-can-language-models-learn-typologically/