Elamur Research⌕
☰
home / papers / 2026 / ManCC: A Task-Anchored Benchmark for Manchu–Classical Chinese Cross-Lingual Modeling

ManCC: A Task-Anchored Benchmark for Manchu–Classical Chinese Cross-Lingual Modeling

Paper · 2026 · Meiqi Wang, Xiaoxin Sun, Dongjie Wang et al.

Summary

Manchu, the official language of the Qing dynasty, is now critically endangered. Translation between Manchu and Classical Chinese has had no shared benchmark, so earlier small studies cannot be compared fairly. The authors build ManCC, which uses bidirectional Manchu–Classical Chinese translation as an anchor task for evaluating historical cross-lingual modeling. Its parallel corpus of 16,627 sentence-aligned pairs comes from the Qing archive Manwen Laodang-Taizong, which had no public bilingual edition. Five Manchu specialists produced the corpus by translating independently, cross-checking each other, and then resolving disagreements. The benchmark adds fixed data splits, automatic scoring with BLEU and chrF, and expert ratings of fidelity, fluency and linguistic normativity on a 1–5 scale. Baselines come from three families: a Transformer trained from scratch, multilingual pretrained models, and LLMs. They show three things. The typological gap between agglutinative Manchu and isolating Classical Chinese shapes results. Pretraining that covers more languages transfers better. Automatic metrics often miss errors that human judges catch. ManCC thus offers a reproducible testbed and a template for other endangered historical languages.

Key points

  • Corpus: 16,627 Manchu–Classical Chinese sentence pairs from Manwen Laodang-Taizong, released via Zenodo for non-commercial academic use.
  • Sentence length: Manchu sentences average about 62 characters (13 words). Classical Chinese sentences average about 17 characters (9.5 words).
  • Evaluation protocol: two tiers, combining automatic BLEU and chrF with 1–5 human ratings on Fidelity, Fluency and Linguistic Normativity. Inter-rater agreement is reported.
  • Baselines: a non-pretrained Transformer, multilingual pretrained models (M2M100, NLLB) and LLMs (e.g., DeepSeek-V3). Broader multilingual coverage helped low-resource transfer.
  • Limitations: all data comes from a single source, so genre diversity is narrow. Expert annotation is costly and hard to scale. Judging normativity and historical style is partly subjective.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Science and Technology of ChinaHefei · China108 works · 112,698 citations🎓Sichuan UniversityChengdu · China14 works · 1,293 citations🎓University of KansasLawrence · United States8 works · 32 citations🎓Northeast Normal UniversityChangchun · China2 works · 1 citation

Also on the byline: Jiangsu Second Normal University · Tonghua Normal University · Changchun Humanities and Sciences College.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 2Research subjects and languages the work is about.Evaluation & benchmarks · 1,220 worksMultilingual & cross-lingual NLP · 828 works

Cite

Cite as Meiqi Wang, Xiaoxin Sun, Dongjie Wang, Ruixin Yu, Xiantao Heng, Shuo Wang, Zhen Huang, Peng Zhao, Suhua Wang and Minghao Yin. 2026. ManCC: A Task-Anchored Benchmark for Manchu–Classical Chinese Cross-Lingual Modeling. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1359

@inproceedings{wang2026mancc,
  title = {{ManCC: A Task-Anchored Benchmark for Manchu–Classical Chinese Cross-Lingual Modeling}},
  author = {Meiqi Wang, Xiaoxin Sun, Dongjie Wang, Ruixin Yu, Xiantao Heng, Shuo Wang, Zhen Huang, Peng Zhao, Suhua Wang and Minghao Yin},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1359},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1359}
}

References

33 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Xiaojun Bi, Wenhao Tao, Zheng Chen, and Haipeng Sun. 2025. Scc3: a novel structure-connected cognition cube network for manchu word recognition. Expert Systems with Applications, page 129374. find
  2. Andong Chen, Lianzhang Lou, Kehai Chen, Xuefeng Bai, Yang Xiang, Muyun Yang, Tiejun Zhao, and Min 27280 10 find
  3. Zhang. 2025. Benchmarking LLMs for translating classical Chinese poetry: Evaluating adequacy, fluency, and elegance. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 33007–33024, Suzhou, China. Association for Computational Linguistics.
  4. Yan Hon Michael Chung and Donghyeok Choi. 2025. Finetuning vision-language models as ocr systems for low-resource languages: A case study of manchu. arXiv preprint arXiv:2507.06761. — [arXiv:2507.06761](https://arxiv.org/abs/2507.06761) find
  5. Pamela Kyle Crossley and Evelyn S. Rawski. 1993. A profile of the manchu language in ch’ing history. Harvard Journal of Asiatic Studies, 53:63. find
  6. Bonaventure F. P. Dossou, Ines Arous, and Jackie CK Cheung. 2025. Rethinking full finetuning from pre-training checkpoints in active learning for African languages. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop), pages… find
  7. Muhammad ElNokrashy and Tom Kocmi. 2023. eBLEU: Unexpectedly good machine translation evaluation using simple word embeddings. In Proceedings of the Eighth Conference on Machine Translation, pages 746–750, Singapore. Association for Computational Linguistics. find
  8. Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, and 1 others. 2021. Beyond english-centric multilingual machine translation. Journal of Machine Learning Research, 22(107):1–48. find
  9. Jiaxin Guo, C. L. Philip Chen, Shuzhen Li, and Tong Zhang. 2024. DEUCE: Dual-diversity enhancement and uncertainty-awareness for cold-start active learning. Transactions of the Association for Computational Linguistics, 12:1736–1754. find
  10. Barathi Ganesh Hb and Michal Ptaszynski. 2025. RBG-AI: Benefits of multilingual language models for low-resource languages. In Proceedings of the Tenth Conference on Machine Translation, pages 1233–1239, Suzhou, China. Association for Computational Linguistics. find
  11. Yepai Jia, Yatu Ji, Xiang Xue, Lei Shi, Qing-Dao-Er-Ji Ren, Nier Wu, Na Liu, Chen Zhao, and Fu Liu. 2025. A semantic uncertainty sampling strategy for back-translation in low-resources neural machine translation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguist… find
  12. Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. The state and fate of linguistic diversity and inclusion in the NLP world. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282–6293, Online. Association for Comp…
  13. Marcin Junczys-Dowmunt, Roman Grundkiewicz, Tomasz Dwojak, Hieu Hoang, Kenneth Heafield, Tom Neckermann, Frank Seide, Ulrich Germann, Alham Fikri Aji, Nikolay Bogoychev, and 1 others. 2018. Marian: Fast neural machine translation in c++. arXiv preprint arXiv:1804.00344.
  14. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. find
  15. Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020. Multilingual denoising pre-training for neural machine translation. Transactions of the Association for Computational Linguistics, 8:726–742. find
  16. Carter Louchheim, Denis Sotnichenko, Yukina Yamaguchi, and Mark Hopkins. 2025. Using encipherment to isolate conditions for the successful fine-tuning of massively multilingual translation models. In Proceedings of the Tenth Conference on Machine Translation, pages 241–252, Suzhou, China. Associatio… find
  17. Christopher Moseley. 2010. Atlas of the World’s Languages in Danger. Unesco. find
  18. Sebastian Nehrdich, Avery Chen, Marcus Bingenheimer, Lu Huang, Rouying Tang, Xiang Wei, Leijie Zhu, and Kurt Keutzer. 2025. MITRA-zh-eval: Using a buddhist Chinese language evaluation dataset to assess machine translation and evaluation metrics. In Proceedings of the 5th International Conference on… find
  19. Jerry Norman. 2003. The manchus and their language (presidential address). Journal of the American Oriental Society, 123(3):483–491. find
  20. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational…
  21. Renhao Pei, Yihong Liu, Peiqin Lin, François Yvon, and Hinrich Schuetze. 2025. Understanding in-context machine translation for low-resource languages: A case study on Manchu. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 87… find
  22. Maja Popović. 2015. chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for Computational Linguistics.
  23. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  24. Sebastian Ruder, Anders Søgaard, and Ivan Vulić. 2019. Unsupervised cross-lingual representation learning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts, pages 31–38. find
  25. Armel Randy Zebaze, Benoît Sagot, and Rachel Bawden. 2025. TopXGen: Topic-diverse parallel data generation for low-resource machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 22358–22381, Suzhou, China. Association for Computational Linguistics. find
  26. Jean Seo, Sungjoo Byun, Minha Kang, and Sangah Lee. 2023. Mergen: The first Manchu-Korean machine translation model trained on augmented data. In Proceedings of the 3rd Workshop on Multi-lingual Representation Learning (MRL), pages 118–124, Singapore. Association for Computational Linguistics. find
  27. Peng Shu, Junhao Chen, Zheng Liu, Hui Wang, Zihao Wu, Tianyang Zhong, Yiwei Li, Huaqin Zhao, Hanqi Jiang, Yi Pan, Yifan Zhou, Constance Owl, Xiaoming Zhai, Ninghao Liu, Claudio Saunt, and Tianming Liu. 2024. Transcending language boundaries: Harnessing llms for low-resource language translation. ArX… — [arXiv:2411.11295](https://arxiv.org/abs/2411.11295) find
  28. Aaron Daniel Snowberger and Choong Ho Lee. 2024. Manchu script letters dataset creation and labeling. Journal of Information & Communication Convergence Engineering, 22(1). find
  29. NLLB Team, Marta R Costa-Jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, and 1 others. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672. find
  30. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30.
  31. Zhiwei Wang, Siyang Lu, Mingquan Wang, Xiang Wei, and Yingjun Qi. 2022. Amre: An attention-based crnn fornbsp;manchu word recognition onnbsp;anbsp;woodblock-printed dataset. In Neural Information Processing: 29th International Conference, ICONIP 2022, Virtual Event, November 22–26, 2022, Proceedings… find
  32. Diandian Zhang, Yan Liu, Zhuowei Wang, and Depei Wang. 2021. Ocr with the deep cnn model for ligature script-based languages like manchu. Scientific programming, 2021(1):5520338. find
  33. Kexun Zhang, Yee Choi, Zhenqiao Song, Taiqi He, William Yang Wang, and Lei Li. 2024. Hire a linguist!: Learning endangered languages in LLMs with in-context linguistic descriptions. In Findings of the Association for Computational Linguistics: ACL 2024, pages 15654–15669, Bangkok, Thailand. Associat… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/wang-2026-mancc-task-anchored-benchmark-manchu/