Elamur Research⌕
☰
home / papers / 2025 / Family Matters: Language Transfer and Merging for Adapting Small LLMs to Faroese

Family Matters: Language Transfer and Merging for Adapting Small LLMs to Faroese

Paper · 2025 · Jenny Kunz, Iben Nyholm Debess and Annika Simonsen

Summary

Small generative models trained mostly on English handle Faroese poorly. Faroese has little text (about 95 million words in deduplicated FineWeb-2) and almost no evaluation data. The authors start from English-only base models and first continue pre-training them on a related Scandinavian language. In some settings they instead combine several such language-specific models through parameter merging. They then continue training on Faroese, comparing updates to all weights with LoRA adapters in the feed-forward layers. To measure the outcome, they build two minimal-pair benchmarks: FoBLiMP for grammatical acceptability and FoBCoMP for text comprehension. Native Faroese linguists also rate generated text. Transfer from a related language proves indispensable, but the most helpful donor depends on the task. Icelandic gives better grammar judgments, while Danish gives better comprehension. LoRA wins on acceptability, whereas full fine-tuning wins on comprehension and summarization. Merging lowers perplexity only under full fine-tuning. The pattern largely repeats for Northern Sámi and Upper Sorbian, giving a practical recipe for other small languages.

Key points

  • Faroese has roughly 70,000 speakers and holds a mixed position among its relatives. Its words and inflection are closest to Icelandic, but its syntax is closer to Mainland Scandinavian.
  • The authors quantified this position with two similarity measures:
    • Lexical similarity was computed on Islex lemma pairs. Icelandic scored highest (0.321), with the other languages at about 0.17–0.21.
    • Syntactic similarity was computed on 2007 Flores-200 sentences tagged with Stanza. All languages scored about 0.79–0.81, with Danish highest (0.814).
  • Two new benchmarks, FoBLiMP and FoBCoMP, are published on the HuggingFace Hub as jekunz/FoBLiMP and jekunz/FoBCoMP.
  • Merging related-language models under full fine-tuning gave the lowest perplexity on Faroese. Merged LoRA adapters were less stable, and gains on the finer probes were inconsistent.
  • The models produce fluent, grammatical Faroese but weak semantic content. Summaries often slip into other languages.
  • The study covers only two model sizes, one LoRA setup and English-only base models with an English-centric tokenizer.

Interesting ideas

  • I1. The authors estimate how close languages are separately for vocabulary and for syntax, using cheap proxies: edit distance over dictionary lemmas and POS-sequence overlap on parallel text. The two axes can point to different "nearest" relatives.
  • I2. Minimal pairs, where one item is correct and the other is a near-identical wrong variant, test grammar and comprehension without needing large labelled sets.
  • I3. Merging models trained on different sources, using weights or task vectors, controls each source's influence more cheaply than retraining on re-weighted mixtures.
  • I4. Grammar quality and comprehension were driven by different donors and different adaptation methods, so one aggregate score would hide real trade-offs.
  • I5. The authors check generality by re-running the whole pipeline on two further unrelated low-resource languages before trusting a surprising result.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (tree): Before UD dialect conversion or training gparse for a new genre, compare POS n-gram profiles across candidate treebanks to choose the best donor data on evidence rather than intuition.
  • I2 (learn): Build minimal-pair suites as negative controls, for example agreement errors or role-swapped sentences, and check that the tagger, parser and frame extractor score the correct member higher.
  • I3 (learn): Merge gparse checkpoints trained on different treebanks or silver-data sources, tuning the mixing weights on a dev set instead of retraining for every data mix.
  • I4 (answer): Track parser accuracy and downstream QA or word-problem accuracy as separate metrics, since the best parser configuration may not give the best readers.
  • I5 (learn): Confirm any unexpected training finding on a second genre or treebank before adopting it as a default.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Linköping UniversityLinköping · Sweden18 works · 2,319 citations🎓University of IcelandReykjavik · Iceland5 works · 41 citations🎓University of the Faroe IslandsTórshavn · Faroe Islands2 works · 2 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksMultilingual & cross-lingual NLP · 828 works

Cite

Cite as Jenny Kunz, Iben Nyholm Debess and Annika Simonsen. 2025. Family Matters: Language Transfer and Merging for Adapting Small LLMs to Faroese. arXiv:2510.00810.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2510.00810 · arXiv 2510.00810

@misc{kunz2025family,
  title = {{Family Matters: Language Transfer and Merging for Adapting Small LLMs to Faroese}},
  author = {Jenny Kunz, Iben Nyholm Debess and Annika Simonsen},
  year = {2025},
  doi = {10.48550/arxiv.2510.00810},
  eprint = {2510.00810},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2510.00810}
}

References

79 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Aakanksha, Arash Ahmadian, Seraphina Goldfarb-Tarrant, Beyza Ermis, Marzieh Fadaee, and Sara Hooker. 2024. Mix data or merge models? optimizing for performance and safety in multilingual contexts. In Neurips Safe Generative AI Workshop 2024. Loubna Ben Allal, Anton Lozhkov, Elie Bakouch, Gabriel Martín Blázquez, Guilherme Penedo, Lewis Tunstall, Andrés Marafioti, Hynek Ky-dlíček, Agustín Piqueres Lajarín, Vaibhav Srivastav, Joshua Lochner, Caleb Fahlgren, Xuan-Son Nguyen, Clémentine Fourrier, Ben Burtenshaw, Hugo Larcher, Haojun Zhao, Cyril Zakka, Mathieu Morlon, and 3 others. 2025. Smollm2: When smol goes big – data-centric training of a small language model. Preprint, arXiv:2502.02737. ↗
  2. Alan Ansell, Edoardo Ponti, Anna Korhonen, and Ivan Vulić. 2022. Composable sparse fine-tuning for cross-lingual transfer. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1778–1796, Dublin, Ireland. Association for Computational Linguistics. find
  3. Dan Biderman, Jacob Portes, Jose Javier Gonzalez Ortiz, Mansheej Paul, Philip Greengard, Connor Jennings, Daniel King, Sam Havens, Vitaliy Chiley, Jonathan Frankle, Cody Blakeney, and John Patrick Cunningham. 2024. LoRA learns less and forgets less. Transactions on Machine Learning Research. Featured Certification. ↗
  4. Tyler A. Chang, Catherine Arnett, Zhuowen Tu, and Ben Bergen. 2024. When is multilinguality a curse? language modeling for 250 high- and low-resource languages. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 4074–4096, Miami, Florida, USA. Association for Computational Linguistics.
  5. Pinzhen Chen, Simon Yu, Zhicheng Guo, and Barry Haddow. 2024. Is it good data for multilingual instruction tuning or just bad multilingual evaluation for large language models? In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 9706–9726, Miami, Florida, USA. Association for Computational Linguistics. find
  6. Leshem Choshen, Elad Venezian, Noam Slonim, and Yoav Katz. 2022. Fusing finetuned models for better pretraining. Preprint, arXiv:2204.03044. ↗
  7. Alexandra Chronopoulou, Dario Stojanovski, and Alexander Fraser. 2023. Language-family adapters for low-resource multilingual neural machine translation. In Proceedings of the Sixth Workshop on Technologies for Machine Translation of Low-Resource Languages (LoResMT 2023), pages 59–72, Dubrovnik, Croatia. Association for Computational Linguistics. find
  8. Marta R Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, and 20 others. 2022. No Language Left Behind: Scaling Human-Centered Machine Translation. Technical report. find
  9. Sara Court and Micha Elsner. 2024. Shortcomings of LLMs for low-resource translation: Retrieval and understanding are both the problem. In Proceedings of the Ninth Conference on Machine Translation, pages 1332–1354, Miami, Florida, USA. Association for Computational Linguistics. ↗
  10. Inger Dagsvold, Snefrid Møllersen, and Vigdis Stordahl. 2015. What can we talk about, in which language, in what way and with whom? Sami patients’ experiences of language choice and cultural norms in mental health treatment. International journal of circumpolar health, 74(1):26952. find
  11. John Dang, Shivalika Singh, Daniel D’souza, Arash Ahmadian, Alejandro Salamanca, Madeline Smith, Aidan Peppin, Sungjin Hong, Manoj Govindassamy, Terrence Zhao, Sandra Kublik, Meor Amer, Viraat Aryabumi, Jon Ander Campos, Yi-Chern Tan, Tom Kocmi, Florian Strub, Nathan Grinsztajn, Yannis Flet-Berliac, and 26 others. 2024. Aya expanse: Combining research breakthroughs for a new multilingual frontier. Preprint, arXiv:2412.04261. ↗
  12. Iben Nyholm Debess. 2017. En undersøgelse af ad-verbialplacering i ledsætninger og deklarativ V1 i færøsk og færødansk. Master thesis, University of Copenhagen. find
  13. Iben Nyholm Debess, Alina Karakanta, and Barbara Scalvini. 2025. What’s wrong with this translation? Simplifying error annotation for crowd evaluation. In Proceedings of the 1st Workshop on Nordic-Baltic Responsible Evaluation and Alignment of Language Models (NB-REAL 2025), pages 42–47, Tallinn, Estonia. The University of Tartu Library. find
  14. Iben Nyholm Debess, Annika Simonsen, and Hafsteinn Einarsson. 2024. Good or bad news? exploring GPT-4 for sentiment analysis for Faroese on a public news corpora. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 7814– 7824, Torino, Italia. ELRA and ICCL. find
  15. Matthew S. Dryer and Martin Haspelmath, editors. 2013. WALS Online (v2020.4). Zenodo. find
  16. Abteen Ebrahimi and Katharina Kann. 2021. How to adapt your pretrained multilingual model to 1600 languages. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4555–4567, Online. Association for Computational Linguistics. ↗
  17. Fahim Faisal and Antonios Anastasopoulos. 2022. Phylogeny-inspired adaptation of multilingual models to new languages. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 434–452, Online only. Association for Computational Linguistics. ↗
  18. Sreyan Ghosh, Chandra Kiran Reddy Evuru, Sonal Kumar, Ramaneswaran S, Deepali Aneja, Zeyu Jin, Ramani Duraiswami, and Dinesh Manocha. 2024. A closer look at the limitations of instruction tuning. In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org. ↗
  19. Kevin Glocker, Kätriin Kukk, Romina Oji, Marcel Bollmann, Marco Kuhlmann, and Jenny Kunz. 2025. Grow up and merge: Scaling strategies for efficient language adaptation. Preprint, arXiv:2512.10772. ↗
  20. Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vladimir Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. 2024. Arcee’s MergeKit: A toolkit for merging large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 477–485, Miami, Florida, US. Association for Computational Linguistics. ↗
  21. Linyang He, Ercong Nie, Sukru Samet Dindar, Arsalan Firoozi, Adrian Florea, Van Nguyen, Corentin Puffay, Riki Shimizu, Haotian Ye, Jonathan Brennan, Helmut Schmid, Hinrich Schütze, and Nima Mesgarani. 2025. Xcomps: A multilingual benchmark of conceptual minimal pairs. Preprint, arXiv:2502.19737. ↗
  22. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  23. Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2023. Editing models with task arithmetic. In The Eleventh International Conference on Learning Representations. ↗
  24. Jógvan ín Lon Jacobsen. 2021. Føroysk Purisma. Fróðskapur, Faroe University Press. find
  25. Jógvan í Lon Jacobsen. 2022. Faroese – the central nordic language? In Marco Battaglia, Alessandro Fambrini, and Anna Wegener, editors, ’Ja, Jeg tæller min troe hver time’. Studi nordici in memoria di Jørgen Stender Clausen, Borealia, Studi di filologia germanica, nederlandistica e scandinavistica, pages 185–204. Pisa University Press, Pisa. find
  26. Kristiina Jokinen, Trung Ngo Trong, and Ville Hautamäki. 2016. Variation in Spoken North Sami Language. In Interspeech, pages 3299–3303. find
  27. Jaap Jumelet, Leonie Weissweiler, and Arianna Bisazza. 2025. Multiblimp 1.0: A massively multilingual benchmark of linguistic minimal pairs. Preprint, arXiv:2504.02768. ↗
  28. Lily Kahn and Riitta-Liisa Valijärvi. 2017. North Sámi: An Essential Grammar. Routledge. find
  29. Jenny Kunz. 2025. Train more parameters but mind their placement: Insights into language adaptation with peft. In Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025), NEALT Proceedings Series, Tallinn, Estonia. ↗
  30. Hynek Kydlíček, Guilherme Penedo, Clémentine Fourier, Nathan Habib, and Thomas Wolf. 2024. Finetasks: Finding signal in a haystack of 200+ multilingual tasks. find
  31. En-Shiun Annie Lee, Sarubi Thillainathan, Shravan Nayak, Surangika Ranathunga, David Ifeoluwa Adelani, Ruisi Su, and Arya D. McCarthy. 2022. Pre-trained multilingual sequence-to-sequence models: A hope for low-resource language translation? In Findings of the Association for Computational Linguistics: ACL 2022, pages 58–67, Dublin, Ireland. Association for Computational Linguistics. find
  32. Zihao Li, Shaoxiong Ji, Hengyu Luo, and Jörg Tiedemann. 2025. Rethinking multilingual continual pretraining: Data mixing for adapting llms across languages and resources. Preprint, arXiv:2504.04152. ↗
  33. Zihao Li, Yucheng Shi, Zirui Liu, Fan Yang, Ali Payani, Ninghao Liu, and Mengnan Du. 2024. Language ranker: A metric for quantifying llm performance across high and low-resource languages. Preprint, arXiv:2404.11553. ↗
  34. Yu-Hsiang Lin, Chian-Yu Chen, Jean Lee, Zirui Li, Yuyan Zhang, Mengzhou Xia, Shruti Rijhwani, Junxian He, Zhisong Zhang, Xuezhe Ma, Antonios Anastasopoulos, Patrick Littell, and Graham Neubig. 2019. Choosing transfer languages for cross-lingual learning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3125–3135, Florence, Italy. Association for Computational Linguistics. ↗
  35. Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016. Assessing the ability of LSTMs to learn syntax-sensitive dependencies. Transactions of the Association for Computational Linguistics, 4:521–535. ↗
  36. Patrick Littell, David R. Mortensen, Ke Lin, Katherine Kairis, Carlisle Turner, and Lori Levin. 2017. URIEL and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 8–14, Valencia, Spain. Association for Computational Linguistics.
  37. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations. on Empirical Methods in Natural Language Processing, pages 875–880, Brussels, Belgium. Association for Computational Linguistics. find
  38. Rebecca Marvin and Tal Linzen. 2018. Targeted syntactic evaluation of language models. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1192–1202, Brussels, Belgium. Association for Computational Linguistics. ↗
  39. Dan Nielsen. 2023. ScandEval: A benchmark for Scandinavian natural language processing. In Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa), pages 185–201, Tórshavn, Faroe Islands. University of Tartu Library. ↗
  40. Sarah McMonagle. 2022. Aspects of language choice online among german-upper sorbian bilingual adolescents. International journal of bilingual education and bilingualism, 25(1):59–79. find
  41. Byung-Doh Oh and William Schuler. 2024. The impact of token granularity on the predictive power of language model surprisal. Preprint, arXiv:2412.11940. ↗
  42. Noah-Manuel Michael and Andrea Horbach. 2025. GermDetect: Verb placement error detection datasets for learners of Germanic languages. In Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025), pages 818–829, Vienna, Austria. Association for Computational Linguistics. find
  43. Guilherme Penedo, Hynek Kydlíček, Vinko Sabolčec, Bettina Messmer, Negar Foroutan, Martin Jaggi, Leandro von Werra, and Thomas Wolf. 2024. Fineweb2: A sparkling update with 1000s of languages. find
  44. Kanishka Misra, Julia Rayz, and Allyson Ettinger. 2023. COMPS: Conceptual minimal pair sentences for testing robust property knowledge and its inheritance in pre-trained language models. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2928–2949, Dubrovnik, Croatia. Association for Computational Linguistics. ↗
  45. Sara Mičkec Mitschke. 2025. Asymmetric societal bilingualism in upper lusatia: Narratives of interlingual upper sorbian-german families. Treatises and Documents, Journal of Ethnic Studies / Razprave in Gradivo, Revija za narodnostna vprašanja, 94(1):77–95. ↗
  46. Christopher Moseley. 2010. Atlas of the World’s Languages in Danger. Unesco. find
  47. Niklas Muennighoff, Alexander M Rush, Boaz Barak, Teven Le Scao, Nouamane Tazi, Aleksandra Piktus, Sampo Pyysalo, Thomas Wolf, and Colin Raffel. 2023. Scaling data-constrained language models. In Thirty-seventh Conference on Neural Information Processing Systems. find
  48. Graham Neubig and Junjie Hu. 2018. Rapid adaptation of neural machine translation to new languages. In Proceedings of the 2018 Conference Hjalmar Páll Petersen. 2010. The Dynamics of Faroese-Danish Language Contact, 1 edition. Germanistische Bibliothek. Universitäsverlag Winter.
  49. Hjalmar Páll Petersen and Caroline Heycock. 2017. The have/be alternation in contemporary faroese. Acta Linguistica Hafniensia, 49(2):143–158. ↗
  50. Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, and Sebastian Ruder. 2020. MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7654–7673, Online. Association for Computational Linguistics.
  51. Sari Pietikäinen. 2008. Sami in the media: Questions of language vitality and cultural hybridisation. Journal of multicultural discourses, 3(1):22–35. ↗
  52. Sari Pietikäinen. 2010. Sámi language mobility: scales and discourses of multilingualism in a polycentric environment. International journal of the sociology of language, 2010(202). ↗
  53. Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton, and Christopher D. Manning. 2020. Stanza: A python natural language processing toolkit for many human languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 101–108, Online. Association for Computational Linguistics. 2025), pages 622–633, Tallinn, Estonia. University of Tartu Library.
  54. Evgeniia Razumovskaia, Ivan Vulić, and Anna Korhonen. 2024. Analyzing and adapting large language models for few-shot multilingual nlu: Are we there yet? Preprint, arXiv:2403.01929. ↗
  55. Reece S Shuttleworth, Jacob Andreas, Antonio Torralba, and Pratyusha Sharma. 2025. LoRA vs full fine-tuning: An illusion of equivalence. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. find
  56. Nathaniel Robinson, Perez Ogayo, David R. Mortensen, and Graham Neubig. 2023. Chat-GPT MT: Competitive for high- (but not low-) resource languages. In Proceedings of the Eighth Conference on Machine Translation, pages 392– 418, Singapore. Association for Computational Linguistics. find
  57. Helge Sandøy. 2005. Sociolinguistic structures chronologically iv: Icelandic and faroese. In Oscar Bandle, Kurt Braunmüller, Ernst Håkon Jahr, Allan Karker, Hans-Peter Naumann, and Ulf Teleman, editors, The Nordic Languages: An International Handbook of the History of the Nordic Languages, Volume 2, pages 1923–1933. Walter de Gruyter, Berlin and New York. find
  58. Barbara Scalvini and Iben Nyholm Debess. 2024. Evaluating the potential of language-family-specific generative models for low-resource data augmentation: A Faroese case study. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 6496–6503, Torino, Italia. ELRA and ICCL. find
  59. Barbara Scalvini, Iben Nyholm Debess, Annika Simonsen, and Hafsteinn Einarsson. 2025a. Rethinking low-resource MT: the surprising effectiveness of fine-tuned multilingual models in the LLM age. In Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025), pages 609–621, Tallinn, Estonia. University of Tartu Library. find
  60. Barbara Scalvini, Annika Simonsen, Iben Nyholm Debess, and Hafsteinn Einarsson. 2025b. Prompt engineering enhances Faroese MT, but only humans can tell. In Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT Annika Simonsen and Hafsteinn Einarsson. 2024. A Human Perspective on GPT-4 Translations: Analysing Faroese to English News and Blog Text Translations. In Proceedings of the 25th Annual Conference of the European Association for Machine Translation (Volume 1), pages 24– 36, Sheffield, UK. European Association for Machine Translation (EAMT). find
  61. Annika Simonsen, Dan Saattrup Nielsen, and Hafsteinn Einarsson. 2025. FoQA: A Faroese question-answering dataset. In Proceedings of the Third Workshop on Resources and Representations for Under-Resourced Languages and Domains (RESOURCEFUL-2025), pages 48–57, Tallinn, Estonia. University of Tartu Library, Estonia. ↗
  62. Dan Saattrup Smart. 2025. Multiwikiqa: A reading comprehension benchmark in 300+ languages. Preprint, arXiv:2509.04111. ↗
  63. Vésteinn Snæbjarnarson, Annika Simonsen, Goran Glavaš, and Ivan Vulić. 2023. Transfer to a low-resource language via close relatives: The case study on Faroese. In Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa), pages 728–737, Tórshavn, Faroe Islands. University of Tartu Library. ↗
  64. George Stoica, Pratik Ramesh, Boglarka Ecsedi, Leshem Choshen, and Judy Hoffman. 2024. Model merging with svd to tie the knots. Preprint, arXiv:2410.19735. ↗
  65. Anke Tang, Li Shen, Yong Luo, Yibing Zhan, Han Hu, Bo Du, Yixin Chen, and Dacheng Tao. 2024. Parameter-efficient multi-task model fusion with partial linearization. In The Twelfth International Conference on Learning Representations. ↗
  66. Höskuldur Thráinsson, Hjalmar Páll Petersen, Jógvan í Lon Jacobsen, and Zakaris Svabo Hansen. 2012. Faroese. An overview and reference grammar, 3 edition. Faroe University Press/Linguistic Institute of Iceland, Tórshavn/Reykjavík. find
  67. A. Torp. 1998. Nordiske språk i nordisk og germansk perspektiv. Novus. find
  68. Alexander Tsvetkov and Alon Kipnis. 2024. Information parity: Measuring and predicting the multilingual capabilities of language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 7971–7989, Miami, Florida, USA. Association for Computational Linguistics. find
  69. Þórdís Úlfarsdóttir. 2014. ISLEX — a multilingual web dictionary. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 2820– 2825, Reykjavik, Iceland. European Language Resources Association (ELRA). find
  70. Cherlon Ussery and Hjalmar P Petersen. 2023. Ditransitives in Faroese: The distribution of IO/DO and PP. In Ditransitives in Germanic Languages: Synchronic and diachronic aspects, pages 299–324. John Benjamins Publishing Company. find
  71. Tu Vu, Aditya Barua, Brian Lester, Daniel Cer, Mohit Iyyer, and Noah Constant. 2022. Overcoming catastrophic forgetting in zero-shot cross-lingual generation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9279–9300, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. find
  72. Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R. Bowman. 2020. BLiMP: The benchmark of linguistic minimal pairs for English. Transactions of the Association for Computational Linguistics, 8:377–392. ↗
  73. Chenxi Whitehouse, Fantine Huot, Jasmijn Bastings, Mostafa Dehghani, Chu-Cheng Lin, and Mirella Lapata. 2024. Low-rank adaptation for multilingual summarization: An empirical study. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 1202–1228, Mexico City, Mexico. Association for Computational Linguistics. find
  74. Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt. 2022. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 23965–23998. PMLR. ↗
  75. Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. 2023. TIES-merging: Resolving interference when merging models. In Thirty-seventh Conference on Neural Information Processing Systems.
  76. Prateek Yadav, Tu Vu, Jonathan Lai, Alexandra Chronopoulou, Manaal Faruqui, Mohit Bansal, and Tsendsuren Munkhdalai. 2024. What matters for model merging at scale? Preprint, arXiv:2410.03617. ↗
  77. Zheng Xin Yong, Hailey Schoelkopf, Niklas Muennighoff, Alham Fikri Aji, David Ifeoluwa Adelani, Khalid Almubarak, M Saiful Bari, Lintang Sutawika, Jungo Kasai, Ahmed Baruwa, Genta Winata, Stella Biderman, Edward Raff, Dragomir Radev, and Vassilina Nikoulina. 2023. BLOOM+1: Adding language support to BLOOM for zero-shot prompting. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11682–11703, Toronto, Canada. Association for Computational Linguistics. ↗
  78. Haobo Zhang and Jiayu Zhou. 2025. Unraveling LoRA interference: Orthogonal subspaces for robust model merging. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 26459–26472, Vienna, Austria. Association for Computational Linguistics. ↗
  79. Ziyu Zhao, Tao Shen, Didi Zhu, Zexi Li, Jing Su, Xuwu Wang, and Fei Wu. 2025. Merging loRAs like playing LEGO: Pushing the modularity of loRA to extremes through rank-wise clustering. In The Thirteenth International Conference on Learning Representations. ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/kunz-2025-family-matters-language-transfer-merging/