Elamur Research⌕
☰
home / papers / 2026 / Grammar as Control: Modular Language Generation for the Long Tail

Grammar as Control: Modular Language Generation for the Long Tail

Paper · 2026 · Ndapa Nakashole

Summary

LLMs could help build language technology for underrepresented languages. Left unguided, though, they produce synthetic data that keeps repeating a few simple sentence patterns and misses much of a language's morphosyntax. Nakashole proposes a prompting approach called Modular Typology-Informed Generation (mTIG), which breaks descriptive grammars down into many small "grammar slices." Each slice targets one phenomenon, such as the passive or causative morphology, and combines family-level rules, language-specific details, a generation instruction and a short example. The LLM uses each slice to produce short English–target sentence pairs. Two optional extra steps help: one fixes words against a bilingual lexicon when one exists, and another adds sentences for vocabulary the corpus has not yet covered. Topic conditioning keeps the content varied. Across three low-resource languages, mTIG produces more typologically balanced corpora than plain prompting or pasting the whole grammar into the prompt. Compact MT models trained on this data even outperform the LLM that generated it. The work matters because descriptive grammars exist for many languages that lack corpora, so this approach could reach far into the long tail.

Key points

  • mTIG raises normalized typological entropy by up to 19% compared with unguided prompting and "grammar dump" baselines.
  • Distilled 1.3B-parameter student MT models beat the teacher LLM by up to +20 chrF, a "student-beats-teacher" effect.
  • On 107 GATITOS languages, frontier LLMs mostly score below a 60 chrF usability threshold for word translation, which is why the lexicon-based correction step is needed.
  • The generation setup uses 3–6 word sentences, 21 topics and a fixed budget of about 300k examples. For each lexicon word still missing from the corpus, it generates 5 extra sentences.
  • Grammar slices transfer across related languages. The Bantu slice library took about 32 hours of manual work to build. Code is available at github.com/okalai-ai/mtig.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of California San DiegoSan Diego · United States73 works · 51,948 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 1Research subjects and languages the work is about.Text generation · 202 works

Cite

Cite as Ndapa Nakashole. 2026. Grammar as Control: Modular Language Generation for the Long Tail. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1725

@inproceedings{nakashole2026grammar,
  title = {{Grammar as Control: Modular Language Generation for the Long Tail}},
  author = {Ndapa Nakashole},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1725},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1725}
}

References

42 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. David Ifeoluwa Adelani, Jesujoba Oluwadara Alabi, Angela Fan, Julia Kreutzer, Xiaoyu Shen, Machel Reid, Dana Ruiter, Dietrich Klakow, Peter Nabende, Ernie Chang, Tajuddeen Gwadabe, Freshia Sackey, Bonaventure F. P. Dossou, Chris Emezue, Colin Leong, Michael Beukman, Shamsuddeen H. Muhammad, Guyo D.… — A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translation
  2. Tatiana Anikina, Jan Cegin, Jakub Simko, and Simon Ostermann. 2025. A rigorous evaluation of LLM data generation strategies for low-resource languages. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 8293–8314, Suzhou, China. Association for Computati…
  3. Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, David Silver, Slav Petrov, Melvin Johnson, Ioannis Antonoglou, Julian Schrittwieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy P. Lillicrap,… — Gemini: A Family of Highly Capable Multimodal Models
  4. Anthropic. 2025. Claude opus 4-5. https://www.anthropic.com/. ↗
  5. Ankur Bapna, Isaac Caswell, Julia Kreutzer, Orhan Firat, Daan van Esch, Aditya Siddhant, Mengmeng Niu, Pallavi Baljekar, Xavier Garcia, Wolfgang Macherey, et al. 2022. Building machine translation systems for the next thousand languages. arXiv preprint arXiv:2205.03983. find
  6. ELCIN Church. 1996. English–Ndonga Dictionary. Oniipa Printing Press. find
  7. Markus Freitag, Ricardo Rei, Nitika Mathur, Chi-kiu Lo, Craig Stewart, Eleftherios Avramidis, Tom Kocmi, George Foster, Alon Lavie, and André F. T. Martins. 2022. Results of WMT22 metrics shared task: Stop using BLEU – neural metrics are better and more robust. In Proceedings of the Seventh Conferen…
  8. Saibo Geng, Martin Josifoski, Maxime Peyrard, and Robert West. 2023. Grammar-constrained decoding for structured NLP tasks without finetuning. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10932– 10952, Singapore. Association for Computational Lingu… find
  9. Nicola Bertoldi and Marcello Federico. 2009. Domain adaptation for statistical machine translation with monolingual resources. In Proceedings of the Fourth Workshop on Statistical Machine Translation, pages 182–189, Athens, Greece. Association for Computational Linguistics.
  10. Malcolm Guthrie. 1935. Lingala grammar and dictionary. (Lingala). find
  11. Wilhelm Heinrich Immanuel Bleek. 1851. De nominum generibus linguarum Africae Australis, Copticae, Semiticarum aliarumque sexualium. Dissertation, Universität zu Bonn. find
  12. Francisco Guzmán, Peng-Jen Chen, Myle Ott, Juan Pino, Guillaume Lample, Philipp Koehn, Vishrav Chaudhary, and Marc’Aurelio Ranzato. 2019. The FLORES evaluation datasets for low-resource machine translation: Nepali–English and Sinhala– English. In Proceedings of the 2019 Conference on Empirical Metho…
  13. Ondřej Bojar and Aleš Tamchyna. 2011. Improving translation model by monolingual data. In Proceedings of the Sixth Workshop on Statistical Machine Translation, pages 330–336, Edinburgh, Scotland. Association for Computational Linguistics. find
  14. Isaac Caswell, Ciprian Chelba, and David Grangier. 2019. Tagged back-translation. In Proceedings of the Fourth Conference on Machine Translation (Volume 1: Research Papers), pages 53–63, Florence, Italy. Association for Computational Linguistics.
  15. Isaac Caswell, Elizabeth Nielsen, Jiaming Luo, Colin Cherry, Geza Kovacs, Hadar Shemtov, Partha Talukdar, Dinesh Tewari, Baba Mamadi Diane, Koulako Moussa Doumbouya, Djibrila Diane, and Solo Farabado Cissé. 2025. Smol: Professionally translated parallel data for 115 under-represented languages. Prep… find
  16. Thera Marie Crane, Karl Lindgren-Streicher, and Andy Wingo. 2004. Hai ti! a beginner’s guide to oshikwanyama. find
  17. DeepSeek-AI. 2024. Deepseek-v3 technical report. Preprint, arXiv:2412.19437. find
  18. Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018. Understanding back-translation at scale. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 489–500, Brussels, Belgium. Association for Computational Linguistics. find
  19. Malcolm Guthrie. 1967. Comparative bantu. an introduction to comparative linguistics and the prehistory of the bantu languages. find
  20. Vu Cong Duy Hoang, Philipp Koehn, Gholamreza Haffari, and Trevor Cohn. 2018. Iterative back-translation for neural machine translation. In Proceedings of the 2nd Workshop on Neural Machine Translation and Generation, pages 18–24, Melbourne, Australia. Association for Computational Linguistics.
  21. Liwei Jiang, Yuanjun Chai, Margaret Li, Mickel Liu, Raymond Fok, Nouha Dziri, Yulia Tsvetkov, Maarten Sap, and Yejin Choi. 2025. Artificial hivemind: The open-ended homogeneity of language models (and beyond). In NeurIPS 2025 Datasets and Benchmarks Track. find
  22. J.K. Kloppers. 1994. English–Rukwangali Dictionary. Gamsberg Macmillan Publishers (Pty.) Ltd. find
  23. Alexander Jones, Isaac Caswell, Orhan Firat, and Ishank Saxena. 2023. GATITOS: Using a new multilingual lexicon for low-resource machine translation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 371–405, Singapore. Association for Computational Lin… find
  24. Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. The state and fate of linguistic diversity and inclusion in the NLP world. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282–6293, Online. Association for Comp…
  25. Tom Kocmi, Christian Federmann, Roman Grundkiewicz, Marcin Junczys-Dowmunt, Hitokazu Matsushita, and Arul Menezes. 2021. To ship or not to ship: An extensive evaluation of automatic metrics for machine translation. In Proceedings of the Sixth Conference on Machine Translation, pages 478–494, Online.… find
  26. Mosh Levy, Alon Jacoby, and Yoav Goldberg. 2024. Same task, more tokens: the impact of input length on the reasoning performance of large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15339–15353, Bangkok, T…
  27. Jouni Filip Maho. 2009. NUGL Online: The online version of the New Updated Guthrie List, a referential classification of the Bantu languages. find
  28. Rita Ramos, Everlyn Asiko Chimoto, Maartje Ter Hoeve, and Natalie Schluter. 2025. GrammaMT: Improving machine translation with grammar-informed in-context learning. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 29920– 29940,…
  29. Felipe Sánchez-Martínez, Víctor M. Sánchez-Cartagena, Juan Antonio Pérez-Ortiz, Mikel L. Forcada, Miquel Esplà-Gomis, Andrew Secker, Susie Coleman, and Julie Wall. 2020. An English-Swahili parallel corpus and its use for neural machine translation in the news domain. In Proceedings of the 22nd Annua… find
  30. Torsten Scholak, Nathan Schucher, and Dzmitry Bahdanau. 2021. PICARD: Parsing incrementally for constrained auto-regressive decoding from language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9895–9901, Online and Punta Cana, Dominican Repu… find
  31. Benjamin Marie, Raphael Rubino, and Atsushi Fujita. 2020. Tagged back-translation revisited: Why does it really work? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5990–5997, Online. Association for Computational Linguistics. find
  32. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Improving neural machine translation models with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 86–96, Berlin, Germany. Association for Computational L… find
  33. Ndapa Nakashole. 2025. Typology-guided adaptation in multilingual models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 21819– 21835, Vienna, Austria. Association for Computational Linguistics. find
  34. Richard Shin, Christopher Lin, Sam Thomson, Charles Chen, Subhro Roy, Emmanouil Antonios Platanios, Adam Pauls, Dan Klein, Jason Eisner, and Benjamin Van Durme. 2021. Constrained language models yield few-shot semantic parsers. In Proceedings of the 2021 Conference on Empirical Methods in Natural La… find
  35. Sebastian Nordhoff and Harald Hammarström. 2011. Glottolog/langdoc: Defining dialects, languages, and language families as collections of resources. In First International Workshop on Linked Science 2011-In conjunction with the International Semantic Web Conference (ISWC 2011). find
  36. OpenAI. 2023. GPT-4 technical report. abs/2303.08774.
  37. CoRR, Edoardo Maria Ponti, Helen O’Horan, Yevgeni Berzak, Ivan Vulić, Roi Reichart, Thierry Poibeau, Ekaterina Shutova, and Anna Korhonen. 2019. Modeling language variation and universals: A survey on typological linguistics for natural language processing. Computational Linguistics, 45(3):559–601.
  38. Maja Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics.
  39. Garrett Tanzer, Mirac Suzgun, Eline Visser, Dan Jurafsky, and Luke Melas-Kyriazi. 2024. A benchmark for learning to translate a new language from one grammar book. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net. find
  40. George Wolfe Robert Tobias and Basil HC Turvey. 1954. English-kwanyama dictionary. Witswatersrand University Press. find
  41. Bailin Wang, Zi Wang, Xuezhi Wang, Yuan Cao, Rif A. Saurous, and Yoon Kim. 2023. Grammar prompting for domain-specific language generation with large language models. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 202… find
  42. Kexun Zhang, Yee Choi, Zhenqiao Song, Taiqi He, William Yang Wang, and Lei Li. 2024. Hire a linguist!: Learning endangered languages in LLMs with in-context linguistic descriptions. In Findings of the Association for Computational Linguistics: ACL 2024, pages 15654–15669, Bangkok, Thailand. Associat… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nakashole-2026-grammar-as-control-modular-language/