Elamur Research⌕
☰
home / papers / 2026 / Multi-Task Learning Trade-offs in Vision–Language Models for Ancient Chinese OCR: An Empirical Analysis of Parameter-Efficient Adaptation

Multi-Task Learning Trade-offs in Vision–Language Models for Ancient Chinese OCR: An Empirical Analysis of Parameter-Efficient Adaptation

Paper · 2026 · Yuhan Shu and Huizi Zhou

Summary

The paper asks whether one vision–language model can handle several ancient Chinese document tasks at once. The setting is the EvaHan2026 benchmark, whose three tasks are OCR of printed text (Task A), analysis of documents with mixed layouts (Task B) and OCR of handwriting (Task C). The authors adapt Qwen2.5-VL-7B-Instruct with LoRA (rank 64), applied only to the language decoder's attention layers; the visual encoder and cross-modal fusion stay frozen. Every task is treated as generation. Tasks A and C produce transcriptions, and Task B produces layout regions as JSON, with no separate detection head. Weighted sampling and prompt guidance target rare characters.

Joint training clearly benefits layout analysis but hurts printed OCR, whose error rate nearly doubles. A Task A–only model shows almost no degradation. The authors therefore attribute the drop to catastrophic forgetting: tasks compete for limited shared adapter capacity, and the layout objective dominates training. Per-task loss curves and a LoRA rank sweep support this reading. The study is a useful cautionary result for anyone using parameter-efficient multi-task adaptation for historical OCR.

Key points

  • Task B (layout): mAP@[.5:.95] reaches 0.2802 versus the Qwen2.5-VL-7B baseline's 0.2006 (+39.6%). Macro F1 rises from 0.1530 to 0.3609 and Avg IoU from 0.6600 to 0.7104.
  • Task A (printed OCR):
    • CER worsens from 0.0618 to 0.1100 (+78% relative).
    • Single-task training on the same data reaches CER 0.0625.
    • Training losses show Task A plateauing and then rising again after roughly step 300, while Task B's loss falls fastest.
  • Task C (handwritten OCR): CER is 0.0963 versus the baseline's 0.0920, a small loss that the paper reports as 0.43 points. Xunzi_Qwen2_VL_7B scores 0.1383.
  • LoRA rank sweep (r = 8–128): Task A CER falls from 0.1450 at r = 8 to 0.0758 at r = 128. The trend is not strictly monotonic: r = 32 (0.1042) beats r = 64 (0.1100).
  • Reported failure modes:
    • Task A in the joint setting confuses rare variant characters, splits complex characters into fragments and loses contextual disambiguation.
    • Task B produced malformed or truncated JSON on about 12% of test samples, which the authors addressed by raising the output token limit from 2048 to 8192.
    • Task B is also weak on text-region granularity (text F1 0.404 on validation).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Wuhan UniversityWuhan · China35 works · 1,127 citations

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksComputer Vision and Pattern Recognition · 2,869 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,320 worksVision & language · 657 works

Cite

Cite as Yuhan Shu and Huizi Zhou. 2026. Multi-Task Learning Trade-offs in Vision–Language Models for Ancient Chinese OCR: An Empirical Analysis of Parameter-Efficient Adaptation. Proceedings of the Fourth Workshop on Language Technologies for Historical and Ancient Languages (LT4HALA 2026) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/5hvjskashyjv

@inproceedings{shu2026multi,
  title = {{Multi-Task Learning Trade-offs in Vision–Language Models for Ancient Chinese OCR: An Empirical Analysis of Parameter-Efficient Adaptation}},
  author = {Yuhan Shu and Huizi Zhou},
  booktitle = {Proceedings of the Fourth Workshop on Language Technologies for Historical and Ancient Languages (LT4HALA 2026) @ LREC 2026},
  year = {2026},
  doi = {10.63317/5hvjskashyjv},
  url = {https://doi.org/10.63317/5hvjskashyjv}
}

References

13 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Galal M. Binmakhashen and Sabri A. Mahmoud. 2019. Document layout analysis: A comprehensive survey. ACM Computing Surveys (CSUR), 52(6):1–36. find
  2. Ning Cheng, Bin Li, Liming Xiao, et al. 2020. Integration of automatic sentence segmentation and lexical analysis of ancient chinese based on BiLSTM-CRF model. In Proceedings of the First Workshop on Language Technologies for Historical and Ancient Languages (LT4HALA), pages 52–58. find
  3. Akash Ghosh et al. 2024. Exploring the frontier of vision-language models: A survey of current methodologies and future directions. arXiv preprint arXiv:2404.07214. — [arXiv:2404.07214](https://arxiv.org/abs/2404.07214) find
  4. Edward J. Hu, Yelong Shen, et al. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR).
  5. Geewook Kim, Teakgyu Hong, et al. 2022. OCR-free document understanding transformer. In European Conference on Computer Vision (ECCV).
  6. Bin Li, Dongbo Wang, Minxuan Feng, et al. 2022. The first international ancient chinese word segmentation and POS tagging bakeoff: Overview of the EvaHan 2022 evaluation campaign. In Proceedings of the Second Workshop on Language Technologies for Historical and Ancient Languages (LT4HALA), pages 1–1… find
  7. Bin Li, Dongbo Wang, et al. 2024. Overview of the EvaHan 2024 shared task on ancient chinese sentence segmentation and punctuation. In Proceedings of the Third Workshop on Language Technologies for Historical and Ancient Languages (LT4HALA), pages 214–221. find
  8. Bin Li et al. 2025. Benchmarking vision-language models on chinese ancient documents: From OCR to knowledge reasoning. arXiv preprint arXiv:2509.09731.
  9. Konstantina Nikolaidou, Mathias Seuret, et al. 2022. A survey of historical document image datasets. arXiv preprint arXiv:2203.08504. — [arXiv:2203.08504](https://arxiv.org/abs/2203.08504) find
  10. Shiyu Shi et al. 2025. Ancient script image recognition and processing: A review. arXiv preprint arXiv:2506.19208. — [arXiv:2506.19208](https://arxiv.org/abs/2506.19208) find
  11. Han Xu, Wang Hongsu, Zhang Sanqian, et al. 2019. Sentence segmentation for classical chinese based on LSTM with radical embedding. The Journal of China Universities of Posts and Telecommunications, 26(2):1–10. 8. find
  12. EvaHan Organizers. 2026. find
  13. EvaHan 2026: Ancient Chinese Vision-Language OCR Dataset. Nanjing Normal University. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/shu-2026-multi-task-learning-trade-offs/