Elamur Research⌕
☰
home / papers / 2026 / Transformers Learning Contrafactives: The Importance of Data Distributions

Transformers Learning Contrafactives: The Importance of Data Distributions

Paper · 2026 · David Strohmaier and Simon Wimmer

Summary

Factive verbs such as know occur across many languages, but no language is known to have a contrafactive: an atomic attitude verb that presupposes its complement is false. Earlier transformer experiments by Strohmaier and Wimmer found contrafactives slightly harder to learn in comprehension but not in production. However, their production setup had two unrealistic features: half of the training targets were untrue ascriptions, and contrafactives were exactly as frequent as factives.

This paper reruns the sequence-to-sequence production task with empirically motivated distributions. These assume a low lying rate, based on Serota et al. (2022), and fewer contrafactives than factives, following Sander (2025). Training sets are sampled with linear programming, and learning trajectories are tested for significance.

The distribution of truth values and verbs does change how fast each verb type is learned. Under the most plausible setting, however, factives and contrafactives are learned equally easily. Where contrafactives lag, they catch up quickly. This weakens learnability-based explanations of why contrafactives are missing from natural languages.

Key points

  • The model maps a main value, a sub value, a mind-world relation (C/I/U) and an attitude content to an output: one of three verbs (factive, contrafactive, non-factive) plus an embedded clause. There are 21 valid value combinations.
  • Distributions combine verb frequency (equal, c-light, c-heavy) with the share of true ascriptions (balanced, t-medium, t-heavy). The t-heavy setting has 93% true targets, reflecting the roughly 7% lying rate reported by Serota et al.
  • In the most plausible setting, c-light/t-heavy, there is no learnability gap and no preference for factives. Where contrafactives do lag, the gap falls mostly within the first 300–400 steps. For example, under c-light/balanced, contrafactive accuracy rises from 47.5% to 99.3% within 80 steps, against 99.6% for factives.
  • T-heavy distributions perform worse once models are fully trained. Equal/t-medium, the setting closest to the earlier 2025 study, performs best. This suggests previous work understated how hard attitude verbs are to learn. Statistics are computed over 250 runs with active dropout.
  • The dataset and code are released at github.com/dstrohmaier/contrafactives_distributions.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 works

Cite

Cite as David Strohmaier and Simon Wimmer. 2026. Transformers Learning Contrafactives: The Importance of Data Distributions. Proceedings of the Third Workshop on the Bridges and Gaps between Formal and Computational Linguistics (BriGap-3).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{strohmaier2026transformers,
  title = {{Transformers Learning Contrafactives: The Importance of Data Distributions}},
  author = {David Strohmaier and Simon Wimmer},
  booktitle = {Proceedings of the Third Workshop on the Bridges and Gaps between Formal and Computational Linguistics (BriGap-3)},
  year = {2026},
  url = {https://aclanthology.org/2026.brigap-1.10/}
}

References

46 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A next-Strohmaier and Wimmer (2025, 406) note that pragmatic-syntactic bootstrapping (see Hacquard and Lidz 2022) might be one such property. find
  2. Strohmaier and Wimmer (2023) implemented a form of pragmatic-syntactic bootstrapping in their comprehension-focused experiments. This property could be explored in future production-focused experiments. generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD Internati… find
  3. Amir Anvari, Mora Maldonado, and Andrés Soria Ruiz. 2019. The puzzle of Reflexive Belief Construction in Spanish. Proceedings of Sinn und Bedeutung, 23(1):57–74. find
  4. Karen Bartsch and Henry M. Wellman. 1995. Children talk about the mind. Oxford University Press, New York. find
  5. James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. 2011. Algorithms for hyper-parameter optimization. In Proceedings of the 25th International Conference on Neural Information Processing Systems, NIPS’11, pages 2546–2554, Red Hook, NY, USA. Curran Associates Inc. find
  6. Tamar Johnson, Kexin Gao, Kenny Smith, Hugh Rabagliati, and Jennifer Culbertson. 2021. Investigating the effects of i-complexity and e-complexity on the learnability of morphological systems. Journal of Language Modelling, 9(1):97–150. Number: 1. find
  7. Julie Kallini, Isabel Papadimitriou, Richard Futrell, Kyle Mahowald, and Christopher Potts. 2024. Mission: Impossible language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, page 14691–14714, Bangkok, Thailand. Association for Computational Lingui…
  8. Gregory Weiss Kierstead. 2015. Projectivity and the Tagalog Reportative Evidential. Master’s thesis, The Ohio State University. find
  9. Manfred Krifka. 2016. Realis and Non-Realis Modalities in Daakie (Ambrym, Vanuatu). Semantics and Linguistic Theory, pages 566–583. find
  10. M. Ryan Bochnak and Emily A. Hanink. 2022. Clausal embedding in Washo: Complementation vs. modification. Natural Language & Linguistic Theory, 40(4):979–1022. find
  11. Mora Maldonado and Jennifer Culbertson. 2019. Learnability as a window into universal constraints on person systems. Proceedings of the Amsterdam Colloquium, 22:484–493. find
  12. Madeline Bossi. 2022. Unifying negative bias and reminding functions: The case of Kipsigis par. In Özge Bakay, Breanna Pratley, Evan Neu, and Peyton Deal, editors, Proceedings of the Fifty-Second Annual Meeting of the North East Linguistic Society, pages 95–104. Amherst. find
  13. Mora Maldonado, Jennifer Culbertson, and Wataru Uegaki. 2022. Learnability and constraints on the semantics of clause-embedding predicates. find
  14. Charlotte Caucheteux and Jean-Rémi King. 2022. Brains and algorithms partially converge in natural language processing. Communications Biology, 5(1). find
  15. Danny Merkx and Stefan L. Frank. 2021. Human Sentence Processing: Recurrence or Attention? In Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, pages 12–22. Association for Computational Linguistics. find
  16. Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. In Proceedings of The 33rd International Conference on Machine Learning, pages 1050–1059. PMLR. find
  17. Valentine Hacquard and Jeffrey Lidz. 2022. On the Acquisition of Attitude Verbs. Annual Review of Linguistics, 8(1):193–212. find
  18. Jack Hoeksema. 2021. Verbs of deception, point of view and polarity. Proceedings of the International Conference on Head-Driven Phrase Structure Grammar, pages 26–46. find
  19. Richard Holton. 2017. I—Facts, Factives, and Contrafactives. Aristotelian Society Supplementary Volume, 91(1):245–266. find
  20. Pei-Yi Katherine Hsiao. 2017. On counterfactual attitudes: a case study of Taiwanese Southern Min. Lingua Sinica, 3(1):4. find
  21. Rachael C. Hulme, Daria Barsky, and Jennifer M. Rodd. 2019. Incidental Learning and Long-Term Retention of New Word Meanings From Stories: The Effect of Number of Exposures. Language Learning, 69(1):18–43. find
  22. William B. McGregor. 2024. On the expression of mistaken beliefs in Australian languages. Linguistic Typology, 28(1):101–145. find
  23. Stuart Mitchell, Michael O’Sullivan, and Iain Dunning. 2011. PuLP: A Linear Programming Toolkit for Python. Preprint, Optimization Online:11731. find
  24. Dario Paape. 2023. When Transformer models are more compositional than humans: The case of the depth charge illusion. Experiments in Linguistic Meaning, 2:202–218. find
  25. Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in PyTorch. find
  26. Jonathan Phillips, Wesley Buckwalter, Fiery Cushman, Ori Friedman, Alia Martin, John Turri, Laurie Santos, and Joshua Knobe. 2020. Knowledge before Belief. Behavioral and Brain Sciences, pages 1–37. find
  27. Tom Roberts and Deniz Özyıldız. 2025. A causal explanation for the contrafactive gap. LingBuzz Published In:. find
  28. Marc Stephen Rosenberg. 1975. Counterfactives: A Pragmatic Analysis of Presupposition. Ph.D. thesis, University of Illinois at Urbana-Champaign. find
  29. Alexis Ross and Ellie Pavlick. 2019. How well do NLI models capture verb veridicality? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2230–2240, Hong Kong, China…
  30. Thorsten Sander. 2020. Fregean Side-Thoughts. Australasian Journal of Philosophy, 0(0). find
  31. Thorsten Sander. 2025. A Puzzle About Anti-Factives. Journal of the American Philosophical Association, pages 1–20. find
  32. Martin Schrimpf, Idan Asher Blank, Greta Tuckute, Carina Kauf, Eghbal A. Hosseini, Nancy Kanwisher, Joshua B. Tenenbaum, and Evelina Fedorenko. 2021. The neural architecture of language: Integrative modeling converges on predictive processing. Proceedings of the National Academy of Sciences, 118(45)… find
  33. Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Eric Larson, and 15 others. 2020. SciPy 1.0: Fundamental algorithms for scientific computing in Python. Nature Methods, 17(3):261–272.
  34. Shuhei Watanabe. 2023. Tree-Structured Parzen Estimator: Understanding Its Algorithm Components and Their Roles for Better Empirical Performance. Preprint, arXiv:2304.11127. — [arXiv:2304.11127](https://arxiv.org/abs/2304.11127) find
  35. Simon Wimmer and David Strohmaier. 2022. Contrafactives and Learnability. In Marco Degano, Tom Roberts, Giorgio Sbardolini, and Marieke Schouwstra, editors, Proceedings of the 23rd Amsterdam Colloquium, pages 298–305. find
  36. Daniel Ziembicki, Karolina Seweryn, and Anna Wróblewska. 2023. Polish natural language inference and factivity: An expert-based dataset and benchmarks. Natural Language Engineering, pages 1–32. find
  37. Kim B. Serota, Timothy R. Levine, and Franklin J. Boster. 2010. The Prevalence of Lying in America: Three Studies of Self-Reported Lies. Human Communication Research, 36(1):2–25. find
  38. Kim B. Serota, Timothy R. Levine, and Tony Docan-Morgan. 2022. Unpacking variation in lie prevalence: Prolific liars, bad lie days, or both? Communication Monographs, 89(3):307–331. find
  39. Shane Steinert-Threlkeld. 2020. An Explanation of the Veridical Uniformity Universal. Journal of Semantics, 37(1):129–144. find
  40. Shane Steinert-Threlkeld and Jakub Szymanik. 2019. Learnability and semantic universals. Semantics and Pragmatics, 12:4:1–39. find
  41. Shane Steinert-Threlkeld and Jakub Szymanik. 2020. Ease of learning explains semantic universals. Cognition, 195. find
  42. Andreas Stokke. 2024. Lies are assertions and presuppositions are not. Inquiry, 0(0):1–24. find
  43. David Strohmaier and Simon Wimmer. 2023. Contrafactives and Learnability: An Experiment with Propositional Constants. Post-Proceedings of Logic and Engineering of Natural Language Semantics 19. find
  44. David Strohmaier and Simon Wimmer. 2025. Contrafactives, Learnability, and Production. Experiments in Linguistic Meaning, 3:395–410. find
  45. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. 31st Conference on Neural Information Processing Systems, pages 1–11.
  46. Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan J. van der Walt, Matthew find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/strohmaier-2026-transformers-learning-contrafactives-importance-data/