📄 ALADAN at IWSLT25 Low-resource Arabic Dialectal Speech Translation Task

Summary

The paper describes the ALADAN team's entry for the IWSLT 2025 low-resource track, which asks for translation of North Levantine Arabic speech into English. The authors kept their cascaded pipeline from the previous year. For ASR, a TDNN-F model and a Zipformer are merged with ROVER, and the Zipformer is now a 253M-parameter Zipformer-Large, trained as one multi-dialect model on about 4200 hours of audio. To make up for scarce parallel text, they worked with the Crowdee platform to collect English translations for five dialects: Tunisian, Levantine, Moroccan, Algerian and Egyptian. Workers had to pass a qualification test, and the output went through several filtering passes. They then fine-tuned NLLB and three LLMs on this data with QLoRA, and Command-R gave the strongest results. When only the crowdsourced data was used, every dialect improved, including an out-of-domain IWSLT test set. When large, high-quality LDC Levantine data was already available, the crowdsourced data brought no further gain. The final document-level system, decoded with MBR, reached 20.0 BLEU on the official test set. The work shows that carefully controlled crowdsourcing can stand in for missing dialect data.

Key points

• ASR WER on the IWSLT24 apc validation set:
  • TDNN-F: 26.5
  • Zipformer-Large: 21.8
  • ROVER combination of both: 19.9
  • Text normalization alone reduced WER by roughly 10% for apc and 18% for aeb.
• Crowdsourced corpus: more than 160k sentence pairs.
  • arq 51.9k, arz 52.8k, ary 19.1k, apc 14.8k, aeb 22.7k.
  • Workers first passed a 40-question test for their dialect.
  • Filtering removed machine-like output, non-English text, length mismatches, and sentences longer than 10 words with GPT-2 perplexity above 100.
• MT models: NLLB 1.3B, Command-R v0.1 (4-bit), Aya Expanse 8B and EuroLLM 9B Instruct, all fine-tuned with QLoRA (r=32, α=16).
• Results with crowd data only:
  • The fine-tuned cascade scored 23.7 BLEU on the IWSLT24 test. The best shared-task system had 28.7 and the runner-up 20.9.
  • On AraBench, fine-tuning lowered NLLB's scores but raised Command-R's.
• Final submission:
  • Document-level Command-R with a context of up to 100 lines.
  • The test set was translated with 29 checkpoints, and MBR decoding using wmt22-comet-da picked the output, for 20.0 BLEU.

Interesting ideas

• I1. Annotators had to pass a skill test before contributing, and each translation came with a self-reported confidence label and optional alternative translations, so every item carries a quality signal from the source.
• I2. Crowd output was cleaned by several cheap and independent checks: machine-likeness, language ID, source/target length ratio, and fluency measured by an outside LM's perplexity.
• I3. Spelling variants were merged by combining Word2vec context similarity with a weighted Levenshtein distance, so words are grouped only when they look alike and also occur in similar contexts.
• I4. Two ASR systems with different architectures were combined by ROVER voting, and the merged output beat both, because the systems make different errors.
• I5. Rather than trusting a single model, the team produced outputs from many checkpoints and kept the candidate that a quality metric judged most consistent with the rest (MBR).

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (learn): Each LLM teacher tier could first be scored on a small gold UD/frame quiz and asked to give a confidence level per answer, so silver data is weighted by how reliable its source has proven to be.
• I2 (learn): Silver parses agreed by witness parsers could also go through independent sanity filters, such as tree-shape anomalies or token/arc count mismatches, before they enter training.
• I3 (tokens): The same two-signal test could map variant spellings, typos and British/American forms to one lemma in crate en, without merging distinct words that merely look similar.
• I4 (tree): The perceptron parser and gparse could be merged by voting on each arc, and the points where they disagree could be marked as explainable uncertainty.
• I5 (back): When generating text from the meaning graph, several realizations could be produced and the one closest to the others under a meaning-preserving similarity score chosen.