Elamur Research⌕
☰
home / papers / 2026 / On the Robustness of Morphosyntactic Transformation with Large Language Models: The Case of Quechua Collao

On the Robustness of Morphosyntactic Transformation with Large Language Models: The Case of Quechua Collao

Summary

Large language models struggle to make targeted grammatical edits in low-resource agglutinative languages, where one word can carry person, tense, number and evidentiality together. The authors study this problem for Quechua Collao (Southern Quechua, quz). They started from a normalized Spanish–Quechua parallel corpus compiled from bilingual teaching materials and public texts. Quechua sentences were tagged semi-automatically using surface morpheme patterns, and native speakers then checked the tags. From this, the authors built a set of (source, change, target) triples. In each task, a model receives a sentence plus a structured list of edits and must output the modified sentence with the same meaning. Three instruction-tuned LLMs were tested through in-context learning. The experiments varied the number of examples, used a hierarchical retrieval of examples ranked by morphosyntactic similarity, and optionally added hints about the expected suffixes. Prompt and task design mattered more than parameter count. More examples helped the smaller model most, the larger model stayed steady, and hints helped only some categories. The resource fills a gap, since no structured dataset for this task previously existed for this variety.

Key points

  • Parallel corpus: 22,581 Spanish–Quechua sentence pairs. The Quechua side has fewer tokens (285,605 vs. 449,580) but more unique word forms (73,637 vs. 52,632), which reflects its agglutinative morphology.
  • Transformation dataset: 6,328 instances from 471 source sentences, with 402 distinct change specifications (≈13.44 transformations per source). PERSON (41.9%), POSS (19.8%) and TENSE (17.6%) dominate the change labels.
  • Models: Qwen2.5-14B-Instruct, Mistral-24B-Instruct and GPT-OSS-20B, run locally via LM Studio. Settings were K ∈ {5, 10, 15}, prompts with and without morphological hints, and greedy decoding (temperature 0, top-k 1).
  • Example selection is hierarchical, starting with exact matches on the Change specification and falling back to matches on the primary category. Evaluation uses exact match accuracy and chrF, plus a small native-speaker validation.
  • Mistral was the most consistent across configurations, while Qwen depended more on larger K. Regular, localized categories (PERSON, POSS) were easier than NUMBER or multi-element TYPE changes.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Topics: Low-resource languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.americasnlp-6.12

@inproceedings{pocco2026robustness,
  title = {{On the Robustness of Morphosyntactic Transformation with Large Language Models: The Case of Quechua Collao}},
  author = {Pocco, Pool and Oncevay, Arturo},
  booktitle = {Proceedings of the Sixth Workshop on NLP for Indigenous Languages of the Americas (AmericasNLP)},
  year = {2026},
  doi = {10.18653/v1/2026.americasnlp-6.12},
  url = {https://doi.org/10.18653/v1/2026.americasnlp-6.12}
}

References

12 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, and 29 others. 2023. Qwen technical report. Preprint, arXiv:2309.16609. find our card
  2. Luis Camacho Caballero and Rodolfo Zevallos Salazar. 2020. Lingüística computacional para la revitalización y el poliglotismo. Revista Letras UNMSM, 91(134):184–198. find
  3. Rodolfo Cerrón-Palomino. 2003. Lingüística quechua, 2 edition. Centro de Estudios Regionales Andinos Bartolomé de Las Casas, Cusco. find
  4. Luis Chiruzzo, Pavel Denisov, Alejandro Molina-Villegas, Silvia Fernandez-Sabido, Rolando Coto-Solano, Marvin Agüero-Torales, Aldo Alvarez, Samuel Canul-Yah, Lorena Hau-Ucán, Abteen Ebrahimi, Robert Pugh, Arturo Oncevay, Shruti Rijhwani, Katharina von der Wense, and Manuel Mager. 2024. Findings of t… find
  5. Nonato Rufino Chuquimamani Valer, Oscar Chávez Gonzales, Felix Alain Riveros Paravi-cino, César Jara Luna, Moisés Cárdenas Guzmán, and Melquíades Quintasi Mamani. 2021. Urin qichwa qillqay yachana mayt’u = Manual de escritura quechua sureño. Ministerio de Educación, Lima. find
  6. Beatrice Cueva Medina, Gabriel Fabrizio Tuco Casquino, and José Alfredo Sulla-Torres. 2024. Development of a neural machine translation model optimized with bert for translation from quechua to spanish. In Proceedings of the 22nd LAC-CEI International Multi-Conference for Engineering, Education, and… find
  7. Ona De Gibert, Robert Pugh, Ali Marashian, Raul Vazquez, Abteen Ebrahimi, Pavel Denisov, Enora Rice, Edward Gow-Smith, Juan Prieto, Melissa Robles, Rubén Manrique, Oscar Moreno, Angel Lino, Rolando Coto-Solano, Aldo Alvarez, Marvin Agüero-Torales, John E. Ortega, Luis Chiruzzo, Arturo Oncevay, and 3… find
  8. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… find
  9. Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. The state and fate of linguistic diversity and inclusion in the NLP world. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282–6293, Online. Association for Comp… find our card
  10. Tom Lupicki, Lavanya Shankar, Kaavya Chaparala, and David Yarowsky. 2025. JHU’s submission to the AmericasNLP 2025 shared task on the creation of educational materials for indigenous languages. In Proceedings of the Fifth Workshop on NLP for Indigenous Languages of the Americas (AmericasNLP), pages… find
  11. OpenAI, Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K. Arora, Yu Bai, Bowen Baker, Haiming Bao, Boaz Barak, Ally Bennett, Tyler Bertao, Nivedita Brett, Eugene Brevdo, Greg Brockman, Sebastien Bubeck, Che Chang, and 107 others. 2025. gpt-oss-120b & gpt-oss-2… find our card
  12. Justin Vasselli, Arturo Martínez Peguero, Junehwan Sung, and Taro Watanabe. 2024. Applying linguistic expertise to LLMs for educational material development in indigenous languages. In Proceedings of the 4th Workshop on Natural Language Processing for Indigenous Languages of the Americas (Americas-N… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pocco-2026-robustness-morphosyntactic-transformation-large-language/