DevLake at LoResMT 2026: The Impact of Pre-training and Model Scale on Russian-Bashkir Low-Resource Translation
Summary
This paper reports Team DevLake's entry in the LoResMT 2026 shared task on Russian-to-Bashkir machine translation. It asks whether cheaply adapting a large multilingual model beats fully training a compact one when only a single consumer GPU (RTX 3080, 10GB) is available. The organizers supplied about 1.2M sentence pairs, but the data was noisy. The author scored each pair with the slone/bert-base-multilingual-cased-bak-rus-similarity classifier and kept two subsets: a strict one (score ≥ 0.80, ~486k pairs) and a permissive one (≥ 0.10, ~923k pairs).
Three systems were compared:
- NLLB-200-1.3B-Distilled, tuned with 4-bit QLoRA on the strict subset.
- M2M-100 (418M).
- MarianMT (opus-mt-en-trk, 77M), fully fine-tuned after its vocabulary was extended with missing Bashkir Cyrillic letters.
NLLB already covers Bashkir in pre-training, and it achieved the best CHRF++ despite seeing roughly half as much data. MarianMT, by contrast, produced fluent output whose meaning often drifted. The broader lesson for low-resource MT is that native pre-training coverage and scale outweigh raw data volume, and that such models can be adapted on modest hardware.
Key points
- Official CHRF++ scores: NLLB-200 (1.3B) 52.67, M2M-100 48.80, and MarianMT (77M) 43.15, despite MarianMT being trained on the larger ~923k-pair set.
- Training setups:
- NLLB: nf4 QLoRA with r=64, α=64, dropout 0.1, adapters on all linear projections, 1 epoch at lr 2e-4.
- MarianMT: full FP32 fine-tuning for 3 epochs at lr 5e-5, with a Bashkir language token prepended to source sentences.
- Inference tricks:
- Seven test sentences matched the training corpus exactly, so their reference translations were inserted directly.
- Decoding used beam size 5, no_repeat_ngram_size=3, and repetition penalty 1.2 to suppress repetition loops.
- Qualitative errors: MarianMT's output shifted with punctuation, turning "apple cider" into words about sitting or planting, or into "leafy chair." NLLB consistently produced Алма сидары.
- Released resources and limitations:
- Code is on GitHub and checkpoints are on the Hugging Face Hub (e.g., Voldis/nllb-1.3b-rus-bak).
- Inference with the 1.3B model needs about 2GB VRAM and has higher latency.
- The BERT-based filter may discard rare dialectal Bashkir.
Cite
@inproceedings{tyurin2026devlake,
title = {{DevLake at LoResMT 2026: The Impact of Pre-training and Model Scale on Russian-Bashkir Low-Resource Translation}},
author = {Tyurin, Vyacheslav},
booktitle = {Proceedings for the Ninth Workshop on Technologies for Machine Translation of Low Resource Languages (LoResMT 2026)},
year = {2026},
doi = {10.18653/v1/2026.loresmt-1.18},
url = {https://doi.org/10.18653/v1/2026.loresmt-1.18}
}References
- Marta R. Costa-jussà and 1 others. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672. find
- Tim Dettmers and 1 others. 2023. Qlora: Efficient finetuning of quantized llms. arXiv preprint arXiv:2305.14314. find
- Angela Fan and 1 others. 2021. Beyond english-centric multilingual machine translation. Journal of Machine Learning Research, 22(107):1–48. find
- Edward J Hu and 1 others. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685. find
- Marcin Junczys-Dowmunt and 1 others. 2018. Marian: Fast neural machine translation in C++. In Proceedings of ACL 2018, System Demonstrations. find our card
- Slone Team. 2023. Bert-base-multilingual-cased-bak-rus-similarity. slone/bert-base-multilingual-cased-bak-rus-similarity. find
This page in Markdown → https://md.research.elamur.ai/free/en/papers/tyurin-2026-devlake-loresmt-2026-impact-pre/