Elamur Research⌕
☰
home / papers / 2026 / Thesis Proposal: Diagnosing and Mitigating Semantic Interference in Script-Sharing Low-Resource Language Models: A Case Study on Square Bai Script

Thesis Proposal: Diagnosing and Mitigating Semantic Interference in Script-Sharing Low-Resource Language Models: A Case Study on Square Bai Script

Paper · 2026 · Jingting Zheng and Deyi Xiong

Summary

Square Bai Script (Fangkuai Baiwen) is derived from Chinese characters. Many of its glyphs look like, or are identical to, Chinese ones but mean something different. Models with strong Chinese knowledge therefore tend to read Bai text through Chinese senses. The authors call this semantic interference. It shows up as substituted senses, Chinese leaking into output and inconsistent disambiguation. This thesis proposal argues that current adaptation pipelines struggle with Bai because they assume visible glyph overlap transfers safely, not only because data is scarce.

Three studies are planned:

  • diagnosing interference against matched non-shared controls;
  • comparing adaptation strategies on a modest compute budget, such as prompting with lexical support, tokenizer and vocabulary controls, continued pretraining and PEFT;
  • mapping when overlap helps or harms, while tracking Chinese retention.

As a proposal, it reports no final results. Its value is in treating script overlap as a conditional variable to be measured and in building audited resources for an underserved written minority language.

Key points

  • Main corpus: The Dabenqu corpus has 28,382 Bai–Chinese sentence pairs, validated by experts, and is the main supervised data.
  • Out-of-domain test: A small Shanhuabei epigraphic set (about 80 items, including roof-tile inscriptions) is reserved for stress testing. A versioned PUA-based glyph encoding and normalization pipeline is also in place.
  • Planned benchmark: BaiBench will provide fixed splits, diagnostic strata and minimal contrast sets, plus a Chinese retention suite.
  • Experimental setup: Experiments center on trainable sub-1B models with parameter-efficient updates, using a 0.6B backbone (Qwen3 offers this size). Translation is the primary task, and synthetic data is only an optional late extension.
  • Timeline: Data auditing and the first version of BaiBench are scheduled for 2026 Q2–Q3, and the main Study III experiments for 2026 Q4–2027 Q1. Raw data that cannot be redistributed will be replaced by released derived artifacts.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Tianjin UniversityTianjin · China52 works · 1,128 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 3,894 worksLow-resource languages · 287 works

Cite

Cite as Jingting Zheng and Deyi Xiong. 2026. Thesis Proposal: Diagnosing and Mitigating Semantic Interference in Script-Sharing Low-Resource Language Models: A Case Study on Square Bai Script. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-srw.43

@inproceedings{zheng2026thesis,
  title = {{Thesis Proposal: Diagnosing and Mitigating Semantic Interference in Script-Sharing Low-Resource Language Models: A Case Study on Square Bai Script}},
  author = {Jingting Zheng and Deyi Xiong},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-srw.43},
  url = {https://doi.org/10.18653/v1/2026.acl-srw.43}
}

References

37 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Belen Alastruey, João Maria Janeiro, Alexandre Allauzen, Maha Elbayad, Loïc Barrault, and Marta R. Costa-jussà. 2025. Interference matrix: Quantifying cross-lingual interference in transformer encoders. arXiv preprint arXiv:2508.02256. — [arXiv:2508.02256](https://arxiv.org/abs/2508.02256) find
  2. Tyler A. Chang, Catherine Arnett, Zhuowen Tu, and Benjamin K. Bergen. 2024. Goldfish: Monolingual language models for 350 languages. arXiv preprint arXiv:2408.10441. find
  3. Jiale Chen, Xuelian Dong, Qihao Yang, Wenxiu Xie, and Tianyong Hao. 2025. Can large language models translate spoken-only languages through international phonetic transcription? In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 23420–23435, Suzhou, Chin… find
  4. Jonathan H. Clark, Dan Garrette, Iulia Turc, and John Wieting. 2022. Canine: Pre-training an efficient tokenization-free encoder for language representation. Transactions of the Association for Computational Linguistics, 10:73–91.
  5. John Dang, Shivalika Singh, Daniel D’souza, Arash Ahmadian, Alejandro Salamanca, Madeline Smith, Aidan Peppin, Sungjin Hong, Manoj Govindassamy, Terrence Zhao, Sandra Kublik, Meor Amer, Viraat Aryabumi, Jon Ander Campos, Yi-Chern Tan, Tom Kocmi, Florian Strub, Nathan Grinsztajn, Yannis Flet-Berliac,… — Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontier
  6. Ona de Gibert, Joseph Attieh, Teemu Vahtola, Mikko Aulamo, Zihao Li, Raúl Vázquez, Tiancheng Hu, and Jörg Tiedemann. 2025. Scaling low-resource MT via synthetic data generation with LLMs. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 27674–27692, Su…
  7. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. QLoRA: Efficient finetuning of quantized LLMs. In Advances in Neural Information Processing Systems.
  8. Peter Devine, Mardhiyah Sanni, Farid Adilazuarda, Julieta Gil Loizaga, and Barry Haddow. 2026. Kakugo: Distillation of low-resource languages into small language models. arXiv preprint arXiv:2601.14051. find
  9. Tianyu Dong, Bo Li, Jinsong Liu, Shaolin Zhu, and Deyi Xiong. 2025. MLAS-LoRA: Language-aware parameters detection and LoRA-based knowledge transfer for multilingual machine translation. In Proceedings of the 63rd Annual Meeting of the Association for Benedikt Ebing, Lennart Keller, and Goran Glavaš… find
  10. Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. 2020. Don’t stop pretraining: Adapt language models to domains and tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8342–8360, Online. A… find
  11. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  12. Jaavid Aktar Husain, Raj Dabre, Aswanth Kumar, Jay Gala, Thanmay Jayakumar, Ratish Puduppully, and Anoop Kunchukuttan. 2024. RomanSetu: Efficiently unlocking multilingual capabilities of large language models via romanization. In Proceedings of the 62nd Annual Meeting of the Association for Computat… find
  13. Renren Jin and Deyi Xiong. 2022. Informative language representation learning for massively multilingual neural machine translation. In Proceedings of the 29th International Conference on Computational Linguistics, pages 5158–5174, Gyeongju, Republic of Korea. International Committee on Computationa… find
  14. Julie Kallini, Dan Jurafsky, Christopher Potts, and Martijn Bartelds. 2025. False friends are not foes: Investigating vocabulary overlap in multilingual language models. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 21138–21154, Suzhou, China. Association for Comput… find
  15. Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, and Nicholas Carlini. 2022. Deduplicating training data makes language models better. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),… find
  16. Tomasz Limisiewicz, Jiří Balhar, and David Mareček. 2023. Tokenization impacts multilingual language modeling: Assessing vocabulary allocation and overlap across languages. In Findings of the Association for Computational Linguistics: ACL 2023, pages 5661–5681, Toronto, Canada. Association for Com… find
  17. Arijit Nag, Soumen Chakrabarti, Animesh Mukherjee, and Niloy Ganguly. 2025. Efficient continual pre-training of LLMs for low-resource languages. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies… find
  18. Artidoro Pagnoni, Ramakanth Pasunuru, Pedro Rodriguez, John Nguyen, Benjamin Muller, Margaret Li, Chunting Zhou, Lili Yu, Jason E. Weston, Luke Zettlemoyer, Gargi Ghosh, Mike Lewis, Ari Holtzman, and Srini Iyer. 2025. Byte latent transformer: Patches scale better than tokens. In Proceedings of the 6… find
  19. Leiyu Pan, Supryadi, and Deyi Xiong. 2023. Is robustness transferable across languages in multilingual neural machine translation? In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 14114–14125, Singapore. Association for Computational Linguistics. find
  20. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational…
  21. Renhao Pei, Yihong Liu, Peiqin Lin, François Yvon, and Hinrich Schuetze. 2025. Understanding in-context machine translation for low-resource languages: A case study on Manchu. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 87… find
  22. Maja Popović. 2017. chrF++: words helping character n-grams. In Proceedings of the Second Conference on Machine Translation, pages 612–618, Copenhagen, Denmark. Association for Computational Linguistics.
  23. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186– 191, Brussels, Belgium. Association for Computational Linguistics.
  24. Ricardo Rei, Craig Stewart, Ana C. Farinha, and Alon Lavie. 2020. COMET: A neural framework for MT evaluation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685–2702, Online. Association for Computational Linguistics.
  25. Oscar Sainz, Jon Campos, Iker García-Ferrero, Julen Etxaniz, Oier Lopez de Lacalle, and Eneko Agirre. 2023. NLP evaluation in trouble: On the need to measure LLM data contamination for each benchmark. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10776–10787, Singap…
  26. Seyoung Song, Haneul Yoo, Jiho Jin, Kyunghyun Cho, and Alice Oh. 2025. Shared heritage, distinct writing: Rethinking resource selection for east asian historical documents. In Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pac… find
  27. Ahmet Üstün, Viraat Aryabumi, Zheng-Xin Yong, Wei-Yin Ko, Daniel D’souza, Gbemileke Onilude, Neel Bhandari, Shivalika Singh, Hui-Lee Ooi, Amr Kayid, Freddie Vargus, Phil Blunsom, Shayne Longpre, Niklas Muennighoff, Marzieh Fadaee, Julia Kreutzer, and Sara Hooker. 2024. Aya model: An instruction fine…
  28. Feng Wang. 2004. On the features of the ancient Bai script. Journal of Dali College, 3(4):9–12. In Chinese. find
  29. Feng Wang. 2020. Types and transmission characteristics of Dabenqu libretto manuscripts in square Bai script. Studies of Ethnic Literature, 38(6):112–120. In Chinese. find
  30. Hao Wang, Hirofumi Shimizu, and Daisuke Kawahara. 2023. Kanbun-lm: Reading and translating classical chinese in japanese methods by language models. In Findings of the Association for Computational Linguistics: ACL 2023, pages 8589–8601, Toronto, Canada. Association for Computational Linguistics. find
  31. Linting Xue, Aditya Barua, Noah Constant, Rami Al-Rfou, Sharan Narang, Mihir Kale, Adam Roberts, and Colin Raffel. 2022. ByT5: Towards a token-free future with pre-trained byte-to-byte models. Transactions of the Association for Computational Linguistics, 10:291–306. find
  32. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388.
  33. Haneul Yoo, Jiho Jin, Juhee Son, JinYeong Bak, Kyunghyun Cho, and Alice Oh. 2022. HUE: Pre-trained model and dataset for understanding hanja documents of ancient korea. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 1832–1844, Seattle, United States. Association for… find
  34. Chen Zhang, Jiuheng Lin, Xiao Liu, Zekai Zhang, and Yansong Feng. 2025. Read it in two steps: Translating extremely low-resource languages with code-augmented grammar books. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3977… find
  35. Jianyu Zheng. 2026. Effective vocabulary expanding of multilingual language models for extremely low-resource languages. arXiv preprint arXiv:2602.09388. — [arXiv:2602.09388](https://arxiv.org/abs/2602.09388) find
  36. Shaolin Zhu, Menglong Cui, and Deyi Xiong. 2024a. Towards robust in-context learning for machine translation with large language models. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 16619–16629, To… find
  37. Shaolin Zhu, Leiyu Pan, Bo Li, and Deyi Xiong. 2024b. LANDeRMT: Dectecting and routing language-aware neurons for selectively finetuning LLMs to machine translation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12135– 12148… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zheng-2026-thesis-proposal-diagnosing-mitigating-semantic/