Elamur Research⌕
☰
home / papers / 2026 / Predict and Fix: A Unified Model for Translation Quality Estimation and Post-Editing

Predict and Fix: A Unified Model for Translation Quality Estimation and Post-Editing

Summary

In localization pipelines, quality estimation (QE) and automatic post-editing (APE) usually run as two separate models. This means the error signals found by the QE model are discarded before correction begins, and each segment needs two expensive inference passes. The authors merge both tasks into one small decoder-only model, Qwen2.5-0.5B. They add a special <QE> token whose final hidden state feeds a two-layer sigmoid regression head. The head predicts a quality score defined as one minus the normalized edit distance between the MT output and the linguist's correction. The standard language-modeling head then emits a PE/No PE decision and, when needed, a revised translation. Both objectives are trained together with uncertainty-weighted multi-task loss. Training uses proprietary, manually post-edited data for eight English-source language pairs at three scales. The largest model gives far better quality scores and edit decisions than zero-shot GPT-5.4 and Gemini 3.1 Pro, and it edits good translations less often. This suggests that task-specific joint training matters more than raw model size for QE-guided post-editing.

Key points

  • Training data: three sets of about 100K, 1M and 1.84M segments covering eight pairs (EN→Canadian French, Japanese, Spanish, Korean, Simplified Chinese, German, Polish, European French).
    • The 1.84M set is rebalanced to roughly 250K segments per pair, except European French (93,364).
    • The test set has 9,048 segments.
  • Main results:
    • The 1.84M model reaches Pearson r = 0.907, MAE 0.054 and PE decision accuracy of 88.4%.
    • The 1M model reaches r = 0.863 and 87.7% accuracy.
    • GPT-5.4 scores r = 0.236 with 65.8% accuracy; Gemini 3.1 Pro scores r = 0.310 with 61.3%.
  • Ablation baseline: an autoregressive Qwen variant that outputs the score as text, without the regression head, isolates the head's contribution. It was trained only on the 100K split.
  • Over-editing remains a problem:
    • EN→Spanish and EN→European French show negative ΔBLEU and ΔCOMET for every system, because the original MT is already strong (BLEU 82.56 and 61.15).
    • Accuracy on No-PE segments is only 77.7%.
  • Limitations:
    • The data is proprietary and cannot be released.
    • The comparison with commercial LLMs is asymmetric, since they ran zero-shot without in-domain data.
    • Future work proposes few-shot baselines and MQM-style error-span prediction.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages · Machine translation

Cite

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{modrzejewski2026predict,
  title = {{Predict and Fix: A Unified Model for Translation Quality Estimation and Post-Editing}},
  author = {Modrzejewski, Maciej and Bhaskar, Yash and Pateria, Chinmay},
  booktitle = {Proceedings of the 17th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track)},
  year = {2026},
  url = {https://aclanthology.org/2026.amta-research.5/}
}

References

20 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901. ↗
  2. Chen, P. and Others (2023). Iterative translation refinement with large language models. In Proceedings of the 24th Annual Conference of the European Association for Machine Translation (EAMT). ↗ our card
  3. Feng, Y. and Others (2024). TEaR: Improving LLM-based machine translation with self-refinement. In Findings of the Association for Computational Linguistics: NAACL 2024. find
  4. Guerreiro, N. M., Rei, R., and Others (2023). xCOMET: Transparent machine translation evaluation through fine-grained error detection. In Proceedings of the Eighth Conference on Machine Translation (WMT). ↗ our card
  5. Huang, J., Chen, X., Mishra, S., Zheng, H. S., Yu, A. W., Song, X., and Chi, E. H. (2024). Large language models cannot self-correct reasoning yet. In Proceedings of the International Conference on Learning Representations (ICLR). ↗ our card
  6. Juraska, J. and Others (2023). MetricX-23: The google submission to the WMT 2023 metrics shared task. In Proceedings of the Eighth Conference on Machine Translation (WMT). find our card
  7. Kendall, A., Gal, Y., and Cipolla, R. (2018). Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7482–7491. find
  8. Ki, C. and Carpuat, M. (2024). Guiding large language models to post-edit machine translation with error annotations. In Findings of the Association for Computational Linguistics: NAACL 2024. ↗
  9. Kocmi, T., Bawden, R., Bojar, O., Martins, A. F. T., et al. (2023). Large language models are state-of-the-art evaluators of translation quality. In Proceedings of the 24th Annual Conference of the European Association for Machine Translation (EAMT). ↗
  10. Kocmi, T. and Federmann, C. (2023). Large language models are state-of-the-art evaluators of translation quality. In Proceedings of the 24th Annual Conference of the European Association for Machine Translation, pages 193–203. ↗
  11. Koneru, S., Huck, M., Exel, M., and Niehues, J. (2025). Quality-aware decoding: Unifying quality estimation and decoding. In Proceedings of the 22nd International Conference on Spoken Language Translation (IWSLT 2025), pages 33–46. ↗
  12. Proceedings of the 17th Conference of the Association for Machine Translation in the Americas, Québec City, Canada, August 31 - September 2, 2026. Volume 1: Research Track Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t., Rocktäschel, T., et al. (2020). Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33:9459–9474. find
  13. Rei, R., Stewart, C., Farinha, A. C., and Lavie, A. (2020a). Comet: A neural framework for mt evaluation. In Proceedings of the 2020 conference on empirical methods in natural language processing (emnlp), pages 2685–2702. find our card
  14. Qwen, :, Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., Lin, H., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Lin, J., Dang, K., Lu, K., Bao, K., Yang, K., Yu, L., Li, M., Xue, M., Zhang, P., Zhu, Q., Men, R., Lin, R., Li, T., Tang, T., Xia, T., Ren, X., Ren, X., Fan, Y., Su, Y., Zhang, Y., Wan, Y., Liu, Y., Cui, Z., Zhang, Z., and Qiu, Z. (2025). Qwen2.5 technical report. ↗
  15. Rei, R., Stewart, C., Farinha, C., and Lavie, A. (2020b). Unbabel’s participation in the wmt20 metrics shared task. In Proceedings of the Fifth Conference on Machine Translation, Online. Association for Computational Linguistics. find our card
  16. Ranasinghe, T., Orasan, C., and Mitkov, R. (2020). Transquest: Translation quality estimation with cross-lingual transformers. In Proceedings of the 28th international conference on computational linguistics, pages 5070– 5081. find our card
  17. Raunak, V. and Others (2023). Leveraging large language models for automated machine translation post-editing. arXiv preprint arXiv:2305.xxxx. find
  18. Zerva, C., Blain, F., De Souza, J. G., Kanojia, D., Deoghare, S., Guerreiro, N. M., Attanasio, G., Rei, R., Orasan, C., Negri, M., et al. (2024). Findings of the quality estimation shared task at wmt 2024: Are llms closing the gap in qe? In Proceedings of the Ninth Conference on Machine Translation, pages 82–109. find
  19. Zhang, B., Haddow, B., and Birch, A. (2023). Prompting large language model for machine translation: A case study. In International conference on machine learning, pages 41092–41110. PMLR. ↗ our card
  20. Proceedings of the 17th Conference of the Association for Machine Translation in the Americas, Québec City, Canada, August 31 - September 2, 2026. Volume 1: Research Track find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/papers/modrzejewski-2026-predict-fix-unified-model-translation/