📄 Selected-Layer Codec Compression for Compact Speech Translation Models: An IWSLT 2026 English-to-Chinese Submission

Summary

The paper addresses how to shrink a large audio-language model for English-to-Chinese speech translation without losing much translation quality. It is a submission to the constrained IWSLT 2026 Model Compression Shared Task, where only ACL60/60 data may be used. Using Qwen2-Audio-7B-Instruct as a fixed backbone, the author tests a post-hoc, training-free codec. Weights are first quantized, which is the lossy step, and then packed with lossless Zstandard coding. The codec is applied only to the feed-forward projections (gate_proj, up_proj, down_proj), and all other parts of the model stay intact. It is compared with FP16, bitsandbytes int8, a codec applied to every linear layer, and a pruning-plus-codec variant. All systems are scored with COMET on a 416-utterance local split. Limiting compression to the MLP layers keeps quality almost at the FP16 level, while compressing all layers clearly hurts it. The work shows that choosing which layers to compress matters more than how aggressively the whole model is compressed. This gives a cheap alternative to compression pipelines that depend on retraining.

Key points

• The FP16 baseline scores 0.7792 COMET at 14.436 GB. The selected-layer q3 + Zstd variant scores 0.7767 at 10.312 GB and was the primary submission. The selected-layer q2 + Zstd variant scores 0.7764 at 8.422 GB, which is the best trade-off.
• Applying the global codec to all linear layers gives the smallest model (3.628 GB, ratio about 3.98×), but COMET drops to 0.7411. Selected-layer pruning plus codec performs worst, at 0.5871 COMET and 8.020 GB.
• The bitsandbytes Int8 baseline keeps quality well (0.7752 COMET at 9.026 GB). However, it is much slower, at about 2.54 s per item compared with 0.66 s for FP16.
• Evaluation uses Unbabel/wmt22-comet-da (unbabel-comet 2.2.2) with a fixed prompt and max_new_tokens=64, run on one RTX A6000 GPU.
• The official ACL60/60 blind-test COMET for the primary run was 0.339. This was measured under different conditions from the local experiments, so the two results are not directly comparable. Code is available at github.com/alonsopg/iwslt-2026-model-compression.