📄 LLMs as Translator Training Partners: A Multi-Agent Approach

Summary

The paper asks whether a large language model can act as a peer-review partner in translator training rather than as an automatic grader. Its premise is that a useful partner need not always be right. It only needs to spot problems and provoke reflection. Four members of an ATA Chinese Language Division practice group each rendered the same exam-style English passage into Chinese and critiqued one another's anonymized work. GPT-5.4 then reviewed the same segments using MQM-style prompts covering both errors and strengths. Its comments were compared with the trainees' remarks and checked by two human evaluators. DeepSeek-V4 served as an independent judge for human–AI disagreements, and the evaluators audited its rulings where needed. GPT-5.4 matched most trainee judgments and produced more numerous, more analytical comments, but it sometimes inflated severity or mislabeled error types. The second model settled many disagreements. The authors argue that combining an MQM-guided reviewer, human deliberation, and LLM arbitration is a workable basis for AI-assisted translator education, while noting the study is small.

Key points

• Data: Four trainees translated an English-to-Chinese excerpt from a McKinsey piece on Asia's future, comparable to an ATA certification exam passage. Reviews were collected in a shared Google Doc and included both positive and negative comments.
• Agreement: GPT-5.4 agreed with the trainees on 77.8% of the problems they flagged and on 88.9% of the strengths they noted.
• Validation: Two human evaluators verified GPT-5.4's own comments, yielding an F1 score of 0.875.
• Weaknesses: GPT-5.4 sometimes overstated error severity, assigned imprecise error categories, demanded rigid source–target alignment, and praised weak or overly literal wording.
• Arbitration and limits: DeepSeek-V4 resolved a substantial share of disagreements. The authors stop short of claiming GPT-5.4 passes a translation "Turing test" and call for larger datasets, more language pairs, and more domains.