📄 A Systematic Comparison of Parameter-Efficient Fine-Tuning Techniques for Low-Resource Neural Machine Translation: Evidence from Indigenous Languages of the Americas

Summary

Translating Indigenous languages of the Americas is hard because parallel data is scarce. Updating every weight of a large multilingual model is costly and can erase useful pretrained knowledge. Parameter-efficient fine-tuning (PEFT) is a possible alternative, but it had not been compared systematically in this setting. The authors first chose NLLB-200-distilled-600M as the base model over mBART-50 and ByT5-small. They then tested eight PEFT methods (LoRA, AdaLoRA, DoRA, VeRA, IA³, OFT, BOFT, Prefix Tuning) against full fine-tuning on 13 Indigenous-to-Spanish pairs from the AmericasNLP shared-task data. The pairs were grouped into four resource tiers, and each experiment was run with three seeds. Evaluation relied mainly on chrF++, with BLEU as a secondary metric. Orthogonal Finetuning (OFT) matched full fine-tuning in quality without a significant difference, while training under 0.3% of the parameters. It was clearly better when data was very scarce. The study shows that cheap adaptation is a realistic option for underserved languages. It also gives practical advice on which method to choose depending on data size and compute budget.

Key points

• Setup: Training sets range from 357 sentences (Chatino) to 125,008 (Quechua), covering 10 language families. All code and configurations are released on GitHub (peft-nmt-americas).
• Overall results:
  • On the dev set, OFT scored best (26.63 chrF++ with 0.28% of parameters), and LoRA followed closely (25.27 with 0.19%).
  • On the test set, full fine-tuning led only marginally (25.12 vs. 25.06 for OFT), a difference that was not statistically significant.
• Effect of data size:
  • In the very-low tier (<5K sentences), OFT beat full fine-tuning by about 3 chrF++ points.
  • Full fine-tuning came out ahead only in the high tier (>25K sentences).
• Mechanism vs. size: How a method adapts the model matters more than how many parameters it trains.
  • IA³ (74K parameters) outscored VeRA (616K) by 2.33 points.
  • OFT beat AdaLoRA by 2.86 points with a similar parameter budget.
  • VeRA and Prefix Tuning were consistently the weakest methods.
• Error analysis: A single aggregate score hides different failure profiles. Full fine-tuning had the top chrF++ but weak entity preservation (F1 0.484). IA³ ranked 7th in chrF++ yet had the best entity F1 (0.541).