📄 Improving Low-Resource Machine Translation via Cross-Linguistic Transfer from Typologically Similar High-Resource Languages

Summary

The paper asks whether fine-tuning a translation model built for a high-resource language helps a related low-resource language when the pairs come from several different families, not just Romance. The author takes pretrained encoder–decoder transformer models for five high-resource sources, all translating into English: Modern Standard Arabic, Hausa, Spanish, Slovak and Eastern Armenian. Each is fine-tuned on a few thousand parallel sentences of a related low-resource variety: Levantine Arabic, Zulu, Catalan, Macedonian and Western Armenian. Data come from NLLB and Western Armenian corpora. Each pair is trained 4–6 times with different hyperparameters, so the conclusions do not rest on one lucky configuration. Quality is measured with BLEU, a small human score (HES) and a combined score (OES). Transfer improved translation for every pair. Gains were larger when parent and child were more similar, but the method still helped for distant pairs. This matters because it gives a cheap recipe for minority and endangered languages: reuse an existing model, collect a small bitext, and do minimal tuning.

Key points

• Five parent→child pairs, all with English as the target side. English was chosen because native-speaker evaluators were available only for English.
• The paper's family labels are questionable. It calls Armenian a "language isolate" and Hausa "Bantu", but Armenian is usually classed as its own Indo-European branch and Hausa as Chadic (Afroasiatic).
• Learning rates must stay on the order of n × 10⁻⁴ to avoid catastrophic failure. Exact values within that range barely matter.
• About 6–7 epochs are enough, and weight decay had no statistically significant effect.
• Batch size depends on similarity. A medium size such as 32 works best for close pairs, while smaller batches help more distant pairs.
• Stated limitations:
  • The human evaluation is small: three prompts and few raters.
  • Colloquial speech and code-switching are underrepresented in the data.
  • Only encoder–decoder models were tested.
  • Using the standard variety risks favouring dominant dialects.

Interesting ideas

• I1. The study treats hyperparameter variation as a robustness control rather than the object of study, running several configurations per condition before drawing conclusions. This guards against results that come from one lucky setting.
• I2. Parent and child tasks share one side, here English as the target, so the effect of transfer can be isolated from the other changes.
• I3. How fine-tuning should be configured depends on how distant the child is from the parent. More distant targets benefit from smaller batches, which allow finer-grained updates.
• I4. The paper pairs a brittle automatic metric (BLEU) with a small human judgment and merges them into one combined score. This catches cases where surface overlap misleads.
• I5. The paper recommends stratifying evaluation by sub-variety, so that a model tuned on the standard form does not quietly fail on other regional or diasporic forms.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): when comparing a new parser or repair-rule variant against the baseline, report results across a small fixed grid of seeds and settings, so an improvement is not an artifact of one configuration.
• I2 (tree): adapt gparse from a large English UD treebank to a small treebank of another genre or dialect while keeping the annotation scheme fixed, so any gain is attributable to the domain transfer itself.
• I3 (tree): when fine-tuning the transformer parser on treebanks far from its training genre (speech, social media), try smaller batches and few epochs first, instead of reusing the in-domain recipe.
• I4 (back): score text regenerated from the meaning graph with both an overlap metric and a small human or teacher-judged adequacy check, because a paraphrase that preserves meaning can score poorly on n-gram overlap.
• I5 (tokens): report tagger and lemmatizer accuracy separately for British/American spelling and for informal registers, so that a regression on a minority variety is not hidden by the overall average.