📄 Abjad AI at KSAA-2026 Shared Task 2: Grouped Speech Conditioning for Arabic Diacritization

Summary

The paper addresses speech-aware Arabic diacritization for KSAA-2026 Shared Task 2. Given dictated audio and its undiacritized transcript, a system must produce fully vowelized text. Many Arabic words allow several valid vowelizations, and text alone often cannot choose between them, while the audio usually can. The authors build on their earlier CATT-Whisper system. They keep CATT, a character-level encoder-only Transformer, unchanged and inject speech in a simple way: speech-encoder frames are averaged in fixed-size groups, linearly mapped into CATT's embedding space, and prepended to the character sequence. No cross-attention is needed. Training has two phases: first the pretrained text encoder is frozen, then the whole model is fine-tuned. The authors sweep speech backbones (Squeezeformer, and Whisper base, small and large-v3) and group sizes. Whisper-small with groups of 5 frames worked best, reaching DER 5.68 and WER 11.85 on the official test. It outperformed the larger Whisper-large-v3, which suggests that modest speech features suit this lightweight fusion better than heavier ones.

Key points

• The fusion averages every G speech frames into one token (K = T/G), projects these tokens, and concatenates them before the L text tokens. G was tuned over {3, 5, 10, 15}.
• On dev (excluding no-diacritic tokens, without case endings), Whisper-small with G=5 scored DER 6.18 and WER 13.29. Whisper-large-v3 reached at best 6.50 DER. Squeezeformer was the weakest backbone, with DER of about 6.8–7.0 at every G.
• On the official test the system scored DER 5.68 and WER 11.85, placing third on the leaderboard (the top entry had DER 4.80).
• Training used cross-entropy, AdamW, batch size 32 and maximum length 1024. CATT was frozen for 10 epochs and then unfrozen for 10 more. The authors note this schedule was not ablated.
• Preprocessing was minimal: non-Arabic characters were removed and whitespace collapsed. No data augmentation was used. Counting case endings sharply raises the error rates; for example, dev WER rises to 26.14 in the excluded setting.

Interesting ideas

• I1. An extra modality can be added to a pretrained sequence encoder without changing its architecture: compress the side signal into a few projected tokens and simply prepend them to the input.
• I2. The pooling group size acts as an explicit dial between detail and sequence length. Too fine a grouping adds redundant context and compute, while too coarse a grouping blurs short cues, so the dial is tuned on dev.
• I3. Freezing the pretrained backbone while the new projection learns, and only then fine-tuning everything, protects existing representations from early noisy gradients.
• I4. Model selection is sharper when scoring excludes trivial positions and counts only those where a decision is actually needed.
• I5. A mid-sized pretrained encoder beat a much larger one in this light fusion setup, so bigger is not automatically better as a feature source.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tree): gparse could take a handful of projected "context tokens" (for example, a summary of the previous sentence or document genre) prepended to the word sequence, adding context without redesigning the parser.
• I2 (world): when feeding compiled knowledge or entity states into a model, a tunable pooling granularity offers a principled way to trade detail against cost instead of fixing it by hand.
• I3 (learn): when bolting new feature heads (such as frame or role signals) onto a trained parser, a freeze-then-unfreeze schedule could stop silver-data noise from damaging good weights early on.
• I4 (learn): evaluation of taggers and parsers could report scores restricted to genuinely ambiguous tokens, so improvements are not hidden by easy cases.
• I5 (learn): when choosing witness parsers or teacher-model tiers for silver data, it is worth sweeping sizes rather than assuming the largest model gives the most useful signal.