Elamur Research⌕
☰
home / papers / 2025 / Can LLMs Help Create Grammar?: Automating Grammar Creation for Endangered Languages with In-Context Learning

Can LLMs Help Create Grammar?: Automating Grammar Creation for Endangered Languages with In-Context Learning

Paper · 2025 · Piyapath T. Spencer and Nanthipat Kongborrirak

Summary

The paper asks whether large language models can speed up formal grammar writing for endangered languages, a task that is normally slow and expensive manual work. The case study is Moklen, an oral Austronesian language of southern Thailand. The authors use only the resources a field linguist typically has early in documentation: a bilingual Moklen–English dictionary and a small set of parallel sentences. They do not train or fine-tune a model. Instead, they build an in-context learning pipeline modelled on interlinear glossing. Source sentences are tokenised using dictionary entries, each token is mapped to its English sense, and the model is prompted to produce grammar rules and lexical entries in XLE format, the computational implementation of LFG. A deliberately small model was able to capture core structural and lexical properties of Moklen. Problems remain, including English grammatical bias and hallucinations caused by Thai–Moklen script overlap. The authors see the approach as a cheap supporting tool for linguists, not a substitute for their work.

Key points

  • Language: Moklen (ISO 639-3: mkm) is endangered.
    • It has fewer than 1,000 speakers, most of them over 50, living mainly in Phang Nga and Phuket.
    • It has SVO order, nominative-accusative alignment, and no inflectional morphology.
    • It is written in a Thai-based orthography.
  • Inputs and output: The method uses a pilot Moklen dictionary (Pittayaporn et al., 2022), in development since documentation began in late 2017, plus a limited bitext. The output is XLE grammar together with lexical entries.
  • Pipeline: The steps are tokenisation, then sense-mapping, then generation. The work extends the LINGOLLM line of research (Zhang et al., 2024) from translation-style tasks to grammar induction.
  • Model choice: The authors intentionally used a small model to show the method works with modest compute. They expect larger models to handle subtler phenomena better.
  • Limitations:
    • Only one isolating language was tested.
    • Outputs showed English bias.
    • The model sometimes confused Thai words with Moklen ones.
    • The authors call for a meaning-oriented evaluation metric and for tests on typologically diverse languages.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Ludwig-Maximilians-Universität MünchenMunich · Germany107 works · 56,478 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksLow-resource languages · 267 works

Cite

Cite as Piyapath T. Spencer and Nanthipat Kongborrirak. 2025. Can LLMs Help Create Grammar?: Automating Grammar Creation for Endangered Languages with In-Context Learning. Proceedings of the 31st International Conference on Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{spencer2025can,
  title = {{Can LLMs Help Create Grammar?: Automating Grammar Creation for Endangered Languages with In-Context Learning}},
  author = {Piyapath T. Spencer and Nanthipat Kongborrirak},
  booktitle = {Proceedings of the 31st International Conference on Computational Linguistics},
  year = {2025},
  url = {https://aclanthology.org/2025.coling-main.681/}
}

References

14 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Kavita Ganesan. 2018. Rouge 2.0: Updated and improved measures for evaluation of summarization tasks. Preprint, arXiv:1803.01937. — [arXiv:1803.01937](https://arxiv.org/abs/1803.01937) find
  2. Akari Asai, Zexuan Zhong, Danqi Chen, Pang Wei Koh, Luke Zettlemoyer, Hannaneh Hajishirzi, and Wen tau Yih. 2024. Reliable, adaptable, and at-John T. Maxwell and Ronald M. Kaplan. 1993. The interface between phrasal and functional constraints. Computational Linguistics, 19(4):571–590.
  3. Diego Miranda-Saavedra. 2024. Generative ai models and the quest for human-level artificial intelligence. Real World Data Science. find
  4. Stephan Oepen, Kristina Toutanova, Stuart Shieber, Christopher Manning, Dan Flickinger, and Thorsten Brants. 2002. The LinGO redwoods treebank: Motivation and preliminary applications. In COLING 2002: The 17th International Conference on Computational Linguistics: Project Notes.
  5. OpenAI. 2024. Gpt-4 technical report. arXiv:2303.08774.
  6. Preprint, Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Com…
  7. Pittayawat Pittayaporn and Songphan Choemprayong. 2024. A proposal for a thai-based moklen orthography. Language Documentation and Conservation. find
  8. Pittayawat Pittayaporn, Warunsiri Pornpottanamas, and Daniel Loss. 2022. Moklen-Thai-English Dictionary: A Pilot Version. Chulalongkorn University Press. find
  9. Maja Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics.
  10. Bradley L. Pritchett. 1992. Grammatical Competence and Parsing Performance. The University of Chicago Press. find
  11. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022. Finetuned language models are zero-shot learners. Preprint, arXiv:2109.01652.
  12. Olga Zamaraeva, Chris Curtis, Guy Emerson, Antske Fokkens, Michael Goodman, Kristen Howell, T.J. Trimble, and Emily M. Bender. 2022. 20 years of the grammar matrix: cross-linguistic hypothesis testing of increasingly complex interactions. Journal of Language Modelling, 10(1):49–137. find
  13. Kexun Zhang, Yee Choi, Zhenqiao Song, Taiqi He, William Yang Wang, and Lei Li. 2024. Hire a linguist!: Learning endangered languages in LLMs with in-context linguistic descriptions. In Findings of the Association for Computational Linguistics ACL 2024, pages 15654–15669, Bangkok, Thailand and virtua… find
  14. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. Preprint, arXiv:1904.09675. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/spencer-2025-can-llms-help-create-grammar/