Elamur Research⌕
☰
home / papers / 2026 / LGSA: Label Geometry Structuring and Aligning for Hierarchical Text Classification

LGSA: Label Geometry Structuring and Aligning for Hierarchical Text Classification

Paper · 2026 · Shuai Zhang, Weibo Xu, Jiahao Nie et al.

Summary

The paper addresses hierarchical text classification (HTC), where texts are assigned to labels in a deep taxonomy. Prompt-tuning and contrastive methods learn label embeddings only implicitly. The authors argue this distorts the spatial layout of labels in two ways. Sibling labels end up crowded together ("structural confusion"), and rare tail labels get pulled into their parents or head labels ("representation collapse"). Their proposed two-stage framework, LGSA, makes the hierarchy an explicit geometric target.

  1. Initialization (HGS module). BERT word embeddings of the labels are rearranged top-down using a general orthogonal frame (GOF) and Gram-Schmidt orthogonalization. The result is a fixed label prototype in which head labels are orthogonal, siblings differ along orthogonal residual directions, and children point in a direction consistent with their parent.
  2. Training (HGA module). A regularizer combines global and local losses: global orthogonality, parent-child similarity, sibling residual orthogonality, and norm decay. It keeps the learned label embeddings of a prompt-tuning model aligned with the prototype.

On WOS, RCV1-V2 and NYT, LGSA outperforms state-of-the-art baselines at almost no extra cost. Its main conceptual contribution is showing that explicitly constructed, hierarchy-aware label geometry is an effective approach for HTC.

Key points

  • Built on the HPT prompt-tuning model and compared mainly with HPT and SIHTC on WOS, RCV1-V2 and NYT; code is available at github.com/LGSA666/LGSA.
  • Overhead: about 0.1M extra parameters on top of HPT (roughly 0.09% of its size), with identical GPU memory and similar training speed. SIHTC, by contrast, needs around 2000MB more memory on RCV1-V2.
  • Low-resource setting (10% of training data): LGSA beats HPT and SIHTC on every dataset and metric except NYT Macro-F1 (59.28 vs. SIHTC's 59.51). On WOS it reaches 82.36 Micro-F1 and 73.71 Macro-F1.
  • Hierarchical consistency: LGSA scores best on both C-MiF1 and C-MaF1 for RCV1-V2 (87.53 / 70.25) and NYT (79.96 / 69.43).
  • Stated limitation: the rigid orthogonal prototype cannot capture varying semantic closeness among sibling labels. The authors plan dynamic geometry learning as future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Zhejiang University of Finance and EconomicsHangzhou · China2 works

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

Cite

Cite as Shuai Zhang, Weibo Xu, Jiahao Nie and Kecheng Huang. 2026. LGSA: Label Geometry Structuring and Aligning for Hierarchical Text Classification. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.1322

@inproceedings{zhang2026lgsa,
  title = {{LGSA: Label Geometry Structuring and Aligning for Hierarchical Text Classification}},
  author = {Shuai Zhang, Weibo Xu, Jiahao Nie and Kecheng Huang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1322},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1322}
}

References

35 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rami Aly, Steffen Remus, and Chris Biemann. 2019. Hierarchical multi-label classification of text with capsule networks. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 323– 330, Florence, Italy. Association for Computation… find
  2. Fuhan Cai, Duo Liu, Zhongqiang Zhang, Ge Liu, Xiaozhe Yang, and Xiangzhong Fang. 2024a. NER-guided comprehensive hierarchy-aware prompt tuning for hierarchical text classification. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluati… find
  3. Fuhan Cai, Zhongqiang Zhang, Duo Liu, and Xiangzhong Fang. 2024b. COPHTC: Contrastive learning with prompt tuning for hierarchical text classification. In Proceedings of the 2024 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 5400–5404, Seoul, Korea. IEEE. find
  4. Ali Cevahir and Koji Murakami. 2016. Large-scale multi-class and hierarchical product categorization for an E-commerce giant. In Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers, pages 525–535, Osaka, Japan. find
  5. The COLING 2016 Organizing Committee. find
  6. Haibin Chen, Qianli Ma, Zhenxi Lin, and Jiangyue Yan. 2021. Hierarchy-aware label semantics matching network for hierarchical text classification. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Languag… find
  7. Hien Dang, Tho Tran, Tan Nguyen, and Nhat Ho. 2024. Neural collapse for cross-entropy class-imbalanced learning with unconstrained relu features model. In Proceedings of the 41st International Conference on Machine Learning, pages 10017–10040. PMLR. find
  8. Hien Dang, Tho Tran, Stanley Osher, Hung Tran-The, Nhat Ho, and Tan Nguyen. 2023. Neural collapse in deep linear networks: From balanced to imbalanced data. In Proceedings of the 40th International Conference on Machine Learning, pages 6873–6947. PMLR. find
  9. Zhongfen Deng, Hao Peng, Dongxiao He, Jianxin Li, and Philip S. Yu. 2021. HTCInfoMax: A global model for hierarchical text classification via information maximization. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language… find
  10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  11. Jintong Gao, He Zhao, Dandan Guo, and Hongyuan Zha. 2024. Distribution alignment optimization through neural collapse for long-tailed classification. In Proceedings of the 41st International Conference on Machine Learning, pages 14969–14987. PMLR. find
  12. Gene H. Golub and Charles F. Van Loan. 2013. Matrix Computations, 4th edition. The Johns Hopkins University Press, Baltimore, MD. find
  13. Siddharth Gopal and Yiming Yang. 2013. Recursive regularization for large-scale classification with hierarchical and graphical dependencies. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 257–265, Chicago, Illinois, USA. Association for C… find
  14. Ke Ji, Yixin Lian, Jingsheng Gao, and Baoyuan Wang. 2023. Hierarchical verbalizer for few-shot hierarchical text classification. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, pages 2918–2933, Toronto, Canada. Association for Computational Linguistics. find
  15. Gibaeg Kim, SangHun Im, and Heung-Seon Oh. 2024. Hierarchy-aware biased bound margin loss function for hierarchical text classification. In Findings of the Association for Computational Linguistics: ACL 2024, pages 7672–7682, Bangkok, Thailand. Association for Computational Linguistics. find
  16. Fanshuang Kong, Richong Zhang, and Ziqiao Wang. 2025. LH-Mix: Local hierarchy correlation guided mixup over hierarchical prompt tuning. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 636–646, Toronto ON, Canada. Association for Computing Machinery. find
  17. Kamran Kowsari, Donald E. Brown, Mojtaba Heidarysafa, Kiana Jafari Meimandi, Matthew S. Gerber, and Laura E. Barnes. 2017. HDLTex: Hierarchical deep learning for text classification. In Proceedings of the 16th IEEE International Conference on Machine Learning and Applications, pages 364–371, Cancun,…
  18. David D. Lewis, Yiming Yang, Tony G. Rose, and Fan Li. 2004. RCV1: A new benchmark collection for text categorization research. Journal of Machine Learning Research, 5:361–397.
  19. Qitong Liu, Hao Peng, Xiang Huang, Zhifeng Hao, Qingyun Sun, Zhengtao Yu, and Philip S. Yu. 2025. Hierarchical text classification optimization via structural entropy and singular smoothing. IEEE Transactions on Knowledge and Data Engineering, 37(9):5283–5297. find
  20. Yuning Mao, Jingjing Tian, Jiawei Han, and Xiang Ren. 2019. Hierarchical text classification with reinforced label assignment. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 44… find
  21. Vinod Nair and Geoffrey E. Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th International Conference on Machine Learning, pages 807–814, Haifa, Israel. Omnipress. find
  22. Vardan Papyan, X. Y. Han, and David L. Donoho. 2020. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences, 117(40):24652– 24663. find
  23. Thanh Duc Pham, Nam Le Hai, Linh Ngo Van, Diep Thi-Ngoc Nguyen, Sang Dinh, and Thien Huu Nguyen. 2025. Mitigating non-representative prototypes and representation bias in few-shot continual relation extraction. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistic… find
  24. Evan Sandhaus. 2008. The new york times annotated corpus. Linguistic Data Consortium, Philadelphia, PA. LDC2008T19. find
  25. Aixin Sun, Ee-Peng Lim, Wee-Keong Ng, and Jaideep Srivastava. 2004. Blocking reduction strategies in hierarchical text classification. IEEE Transactions on Knowledge and Data Engineering, 16(10):1305– 1308. find
  26. Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph attention networks. In Proceedings of the 6th International Conference on Learning Representations, Vancouver, BC, Canada. OpenReview.net. find
  27. Zihan Wang, Peiyi Wang, Lianzhe Huang, Xin Sun, and Houfeng Wang. 2022a. Incorporating hierarchy into text encoder: A contrastive learning approach for hierarchical text classification. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, pages 7109–7119, Dubli… find
  28. Zihan Wang, Peiyi Wang, Tianyu Liu, Binghuai Lin, Yunbo Cao, Zhifang Sui, and Houfeng Wang. 2022b. HPT: Hierarchy-aware prompt tuning for hierarchical text classification. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3740–3751, Abu Dhabi, United Ar… find
  29. Sishi Xiong, Yu Zhao, Jie Zhang, Mengxiang Li, Zhongjiang He, Xuelong Li, and Shuangyong Song. 2024. Dual prompt tuning based contrastive learning for hierarchical text classification. In Findings of the Association for Computational Linguistics: ACL 2024, pages 12146–12158, Bangkok, Thailand. Assoc… find
  30. Yiming Yang. 1999. An evaluation of statistical approaches to text categorization. Information Retrieval, 1(1):69–90. find
  31. Chao Yu, Yi Shen, and Yue Mao. 2022. Constrained sequence-to-tree generation for hierarchical text classification. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1865–1869, Madrid, Spain. Association for Computing Machinery. find
  32. Simon Chi Lok Yu, Jie He, Víctor Gutiérrez-Basulto, and Jeff Z. Pan. 2023. Instances and labels: Hierarchy-aware joint supervised contrastive learning for hierarchical multi-label text classification. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 8858–8875, Singapor… find
  33. Jie Zhou, Chunping Ma, Dingkun Long, Guangwei Xu, Ning Ding, Haoyu Zhang, Pengjun Xie, and Gongshen Liu. 2020. Hierarchy-aware global model for hierarchical text classification. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1106–1117, Online. Assoc… find
  34. Juncheng Zhou, Lijuan Zhang, Yachen He, Rongli Fan, Lei Zhang, and Jian Wan. 2025. A novel negative sample generation method for contrastive learning in hierarchical text classification. In Proceedings of the 31st International Conference on Computational Linguistics, pages 5645–5655, Abu Dhabi, UAE… find
  35. He Zhu, Junran Wu, Ruomei Liu, Yue Hou, Ze Yuan, Shangzhe Li, Yicheng Pan, and Ke Xu. 2024. HILL: Hierarchy-aware information lossless contrastive learning for hierarchical text classification. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational L… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-lgsa-label-geometry-structuring-aligning/