Elamur Research⌕
☰
home / papers / 2026 / MAST: A Multi-View Alignment Strategy for Optimal Transport-Based Contrastive Clustering of Short Text

MAST: A Multi-View Alignment Strategy for Optimal Transport-Based Contrastive Clustering of Short Text

Paper · 2026 · Zijian Zheng and Yonghe Lu

Summary

Short text clustering is hard because pre-trained sentence encoders are not trained to separate clusters. Existing contrastive clustering frameworks also rely on a single augmented view, and pseudo-labels from the original texts never reach the augmented views, causing noisy and unstable assignments. The authors present MAST, which uses the original-text embedding as an anchor and contrasts it jointly with three other views. The first is an augmented view. The second is a sentence-level view from GCNs run over a cosine-similarity graph of mini-batch sentences. The third is a cluster-level view built from a graph of learnable cluster centers initialized by k-means. Structurally close negatives are down-weighted, and intermediate PLM-layer representations serve as extra negatives to counter over-smoothing. For clustering, Student's t soft assignments on the original view are sharpened using high-confidence instances. Augmented-view memberships are then aligned with optimal transport–derived pseudo-labels from the original view. On six benchmark datasets MAST consistently beats strong baselines, even under semantic sparsity and class imbalance, giving a competitive new reference point for unsupervised short text grouping.

Key points

  • The total loss is a weighted sum of a multi-view contrastive alignment term and a clustering term. The clustering term combines a high-confidence guided loss with an OT-based cross-entropy alignment loss. The best weights were η = 10 and ζ = 1.
  • Final cluster labels come from running k-means on the learned original-view embeddings. Soft memberships during training use a Student's t-distribution, following the DEC approach of Xie et al., 2016.
  • In the sensitivity study on AgNews and SearchSnippets, a temperature of τ = 0.5 worked best. Taking intermediate negatives from the penultimate PLM layer (L−1) was also most effective.
  • Comparisons cover contrastive and pseudo-label clustering methods such as SCCL, RSTC, ProPos, CLSESSP and FNSCC. Encoder baselines include SBERT and BGE-M3.
  • Code is released at github.com/zjzone/MAST. Acknowledged limitations are modest extra compute, fixed loss weights, and fairly coarse graph construction based on cosine similarity and soft assignments.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Sun Yat-sen UniversityGuangzhou · China72 works · 9,502 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

Cite

Cite as Zijian Zheng and Yonghe Lu. 2026. MAST: A Multi-View Alignment Strategy for Optimal Transport-Based Contrastive Clustering of Short Text. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.660

@inproceedings{zheng2026mast,
  title = {{MAST: A Multi-View Alignment Strategy for Optimal Transport-Based Contrastive Clustering of Short Text}},
  author = {Zijian Zheng and Yonghe Lu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.660},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.660}
}

References

40 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Prafulla Bafna, Dhanya Pramod, and Anagha Vaidya. 2016. Document clustering: TF-IDF approach. In 2016 International Conference on Electrical, Electronics, and Optimization Techniques (ICEEOT), pages 61–66. IEEE. find
  2. Nuo Chen, Linjun Shou, Jian Pei, Ming Gong, Bowen Cao, Jianhui Chang, Jia Li, and Daxin Jiang. 2023. Alleviating over-smoothing for unsupervised sentence representation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3552– 35… find
  3. Sihao Chen, Hongming Zhang, Tong Chen, Ben Zhou, Wenhao Yu, Dian Yu, Baolin Peng, Hongwei Wang, Dan Roth, and Dong Yu. 2024. Sub-sentence encoder: Contrastive learning of propositional semantic representations. In Proceedings of the 2024 Conference of the North American Chapter of the Association fo… find
  4. Amir Hadifar, Lucas Sterckx, Thomas Demeester, and Chris Develder. 2019. A self-training approach for short text clustering. In Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), pages 194–199. find
  5. William L. Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 1024–1034.
  6. Zhizhong Huang, Jie Chen, Junping Zhang, and Hongming Shan. 2022. Learning representation for clustering via prototype scattering and positive sampling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(6):7509–7524. find
  7. Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171–4186. Minneapolis, Minnesota.
  8. Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net.
  9. Dongyuan Li, Yusong Wang, Kotaro Funakoshi, and Manabu Okumura. 2023. Joyful: Joint modality fusion and graph contrastive learning for multimoda emotion recognition. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 16051–16069, Singapore. Association f… find
  10. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597–1607. PmLR.
  11. Yiding Lu, Yijie Lin, Mouxing Yang, Dezhong Peng, Peng Hu, and Xi Peng. 2024. Decoupled contrastive multi-view clustering with high-order random walks. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 14193–14201. find
  12. Jinghao Deng, Fanqi Wan, Tao Yang, Xiaojun Quan, and Rui Wang. 2023. Clustering-aware negative sampling for unsupervised sentence representation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 8713–8729, Toronto, Canada. Association for Computational Linguistics. find
  13. Yu Meng, Yunyi Zhang, Jiaxin Huang, Yu Zhang, and Jiawei Han. 2022. Topic discovery via latent space clustering of pretrained language model representations. In Proceedings of the ACM web conference 2022, pages 3143–3152. find
  14. Yasuhiro Fujiwara, Yasutoshi Ida, Sekitoshi Kanai, Atsutoshi Kumagai, and Naonori Ueda. 2021. Fast similarity computation for t-SNE. In 2021 IEEE 37th International Conference on Data Engineering (ICDE), pages 1691–1702. IEEE. find
  15. Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. SimCSE: Simple contrastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894–6910, Online and Punta Cana, Dominican Republic. Association for Computational Linguistic…
  16. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
  17. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar. Association for Computational Linguistics.
  18. Xuan-Hieu Phan, Le-Minh Nguyen, and Susumu Horiguchi. 2008. Learning to classify short and sparse text & web with hidden topics from large-scale data collections. In Proceedings of the 17th International Conference on World Wide Web, pages 91–100. find
  19. Md Rashadul Hasan Rakib, Norbert Zeh, Magdalena Jankowska, and Evangelos Milios. 2020. Enhancement of short text clustering by iterative classification. In Natural Language Processing and Information Systems: 25th International Conference on Applications of Natural Language to Information Systems, p… find
  20. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…
  21. Kaicheng Shen, Ping Li, and Xiao Lin. 2024. Clsessp: Contrastive learning of sentence embedding with strong semantic prototypes. Knowledge-Based Systems, 299:112053. find
  22. Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph attention networks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net. find
  23. Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. 2024. C-pack: Packed resources for general chinese embeddings. In Proceedings of the 47th international ACM SIGIR conference on research and development in information retrieval, pages 641–649.
  24. Junyuan Xie, Ross Girshick, and Ali Farhadi. 2016. Unsupervised deep embedding for clustering analysis. In International Conference on Machine Learning, pages 478–487. PMLR.
  25. Binbin Xu, Jun Yin, and Nan Zhang. 2024. Graph based consistency learning for contrastive multi-view clustering. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 8633–8641. find
  26. Jiahao Xu, Wei Shao, Lihui Chen, and Lemao Liu. 2023a. SimCSE++: Improving contrastive learning for sentence embeddings from two perspectives. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12028–12040, Singapore. Association for Computational Lingui… find
  27. Jiaming Xu, Bo Xu, Peng Wang, Suncong Zheng, Guanhua Tian, and Jun Zhao. 2017. Self-taught convolutional neural networks for short text clustering. Neural Networks, 88:22–31. find
  28. Lingling Xu, Haoran Xie, Zongxi Li, Fu Lee Wang, Weiming Wang, and Qing Li. 2023b. Contrastive learning models for sentence representations. ACM Transactions on Intelligent Systems and Technology, 14(4):1–34. find
  29. Xihong Yang, Cheng Tan, Yue Liu, Ke Liang, Siwei Wang, Sihang Zhou, Jun Xia, Stan Z Li, Xinwang Liu, and En Zhu. 2023. Convert: Contrastive graph clustering with reliable augmentation. In Proceedings of the 31st ACM international conference on multimedia, pages 319–327. find
  30. Yang Yang, Wei Shen, Junfeng Shu, Yinan Liu, Edward Curry, and Guoliang Li. 2025. Cmvc+: a multi-view clustering framework for open knowledge base canonicalization via contrastive learning. IEEE Transactions on Knowledge and Data Engineering, 37(5):2296–2310. find
  31. Jianhua Yin and Jianyong Wang. 2016. A model-based approach for text clustering with outlier detection. In 2016 IEEE 32nd International Conference on Data Engineering (ICDE), pages 625–636. IEEE. find
  32. Dejian Yu and Bo Xiang. 2023. Discovering topics and trends in the field of artificial intelligence: Using lda topic modeling. Expert systems with applications, 225:120114. find
  33. Dejiao Zhang, Feng Nan, Xiaokai Wei, Shang-Wen Li, Henghui Zhu, Kathleen McKeown, Ramesh Nallapati, Andrew O. Arnold, and Bing Xiang. 2021. Supporting clustering with contrastive learning. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Lingui…
  34. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text classification. Advances in Neural Information Processing Systems, 28.
  35. Yanzhao Zhang, Richong Zhang, Samuel Mensah, Xudong Liu, and Yongyi Mao. 2022. Unsupervised sentence representation via contrastive learning with mixing negatives. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 11730–11738. find
  36. Shuai Zhao, Linchao Zhu, Xiaohan Wang, and Yi Yang. 2022. Centerclip: Token clustering for efficient text-video retrieval. In Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval, pages 970–981. find
  37. Hao Zheng, Xinyan Guan, Hao Kong, Wenkai Zhang, Jia Zheng, Weixiang Zhou, Hongyu Lin, Yaojie Lu, Xianpei Han, and Le Sun. 2025a. PPTAgent: Generating and evaluating presentations beyond text-to-slides. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 1…
  38. Xiaolin Zheng, Mengling Hu, Weiming Liu, Chaochao Chen, and Xinting Liao. 2023. Robust representation learning with reliable pseudo-labels generation via self-adaptive optimal transport for short text clustering. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguist… find
  39. Zijian Zheng, Yonghe Lu, and Jian Yin. 2025b. FN-SCC: Fuzzy neighborhood-aware self-supervised contrastive clustering for short text. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 2831–2846, Suzhou, China. Association for Computational Linguistics. find
  40. Nai Zhou, Nianmin Yao, Qibin Li, Jian Zhao, and Yanan Zhang. 2023. Multi-mccr: multiple models regularization for semi-supervised text classification with few labels. Knowledge-Based Systems, 272:110588. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zheng-2026-mast-multi-view-alignment-strategy/