Elamur Research⌕
☰
home / papers / 2026 / CASE – Condition-Aware Sentence Embeddings for Conditional Semantic Textual Similarity Measurement

CASE – Condition-Aware Sentence Embeddings for Conditional Semantic Textual Similarity Measurement

Paper · 2026 · Gaifan Zhang, Yi Zhou and Danushka Bollegala

Summary

The paper addresses Conditional Semantic Textual Similarity (C-STS), where the similarity of two sentences depends on a stated condition (aspect). The authors propose CASE, a two-stage bi-encoder approach built on LLM-based embedding models without retraining them. First, the LLM encodes the condition, while each sentence is placed in the instruction prompt. The sentence thus only shapes the attention over condition tokens during pooling. A condition-only embedding is then subtracted, which makes the embedding space more isotropic and consistently raises C-STS scores. Second, a small supervised feed-forward network maps the embeddings into a much lower-dimensional space, and cosine similarity gives the rating. Training uses the refined C-STS data of Zhang et al. (2025). Prompting the model to encode the condition given the sentence turned out better than the reverse order. LLM embeddings beat MLM embeddings on this task. CASE achieves much of the gain of full fine-tuning at a fraction of the compute. This makes it practical for retrieval, QA answer comparison and KGC-style similarity.

Key points

  • Data: The original C-STS dataset by Deshpande et al. (2023) contains 18,908 instances. Training uses the LLM-refined version by Zhang et al. (2025), which has 14,176 instances.
  • Model size and dimensionality: The projection FFN has fewer than 5M parameters. It compresses embeddings 8×, for example from 4096 (NV-embed-v2, Qwen3-Embedding-8B) down to 512 dimensions.
  • Results with frozen Qwen3-8B (Spearman): The raw embeddings score 24.59, while CASE reaches 71.64.
  • Comparison with LoRA fine-tuning: A LoRA fine-tuned Qwen3-8B reaches 79.91, so CASE gets about 90% of that performance. Adding the FFN on top of the fine-tuned model gives 79.11 at 512 dimensions, which keeps about 99% of the performance.
  • Compute cost: The FFN trains in under a minute on one RTX A6000 GPU using less than 1 GB of memory. LoRA fine-tuning takes more than 4 hours on two A100 GPUs.
  • Limitations and code: The evaluation covers English only. Code is available at github.com/LivNLP/CASE.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

Cite

Cite as Gaifan Zhang, Yi Zhou and Danushka Bollegala. 2026. CASE – Condition-Aware Sentence Embeddings for Conditional Semantic Textual Similarity Measurement. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.eacl-long.231

@inproceedings{zhang2026case,
  title = {{CASE – Condition-Aware Sentence Embeddings for Conditional Semantic Textual Similarity Measurement}},
  author = {Gaifan Zhang, Yi Zhou and Danushka Bollegala},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.231},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.231}
}

References

27 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Joshua Ainslie, Tao Lei, Michiel de Jong, Santiago Ontanon, Siddhartha Brahma, Yury Zemlyanskiy, David Uthus, Mandy Guo, James Lee-Thorp, Yi Tay, Yun-Hsuan Sung, and Sumit Sanghai. 2023. CoLT5: Faster long-range transformers with conditional computation. In Proceedings of the 2023 Conference on Empi…
  2. Ameet Deshpande, Carlos Jimenez, Howard Chen, Vishvak Murahari, Victoria Graf, Tanmay Rajpurohit, Ashwin Kalyan, Danqi Chen, and Karthik Narasimhan. 2023. C-STS: Conditional semantic https://huggingface.co/models textual similarity. In Proceedings of the 2023 Conference on Empirical Methods in Natur… ↗
  3. Samantha Durdy, Michael W. Gaultois, Vladimir Gusev, Danushka Bollegala, and Matthew J. Rosseinsky. 2023. Metrics for quantifying isotropy in high dimensional unsupervised clustering tasks in a materials context. Preprint, arXiv:2305.16372. — [arXiv:2305.16372](https://arxiv.org/abs/2305.16372) find
  4. Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. SimCSE: Simple contrastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894–6910, Stroudsburg, PA, USA. Association for Computational Linguistics.
  5. David Ha, Andrew M Dai, and Quoc V Le. 2017. Hyper-Networks. In International Conference on Learning Representations. find
  6. Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R. Salakhutdinov. 2012. Improving neural networks by preventing co-adaptation of feature detectors. Preprint, arXiv:1207.0580. find
  7. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.
  8. Masahiro Kaneko and D Bollegala. 2021. Unmasking the mask - evaluating social biases in masked language models. National Conference on Artificial Intelligence, pages 11954–11962. find
  9. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. Int Conf Learn Represent, abs/1412.6980.
  10. Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, et al. 2022. Matryoshka representation learning. In Advances in Neural Information Processing Systems. find
  11. Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2024. Nv-embed: Improved techniques for training llms as generalist embedding models. arXiv preprint arXiv:2405.17428.
  12. Baixuan Li, Yunlong Fan, and Zhiqiang Gao. 2024. SEAVER: Attention reallocation for mitigating distractions in language models for conditional semantic textual similarity measurement. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 78–95, Stroudsburg, PA, USA. Associa… find
  13. Xianming Li and Jing Li. 2024. BeLLM: Backward dependency enhanced large language model for sentence embeddings. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 792–804, M… find
  14. Ziyong Lin, Quansen Wang, Zixia Jia, and Zilong Zheng. 2024. Varying sentence representations via condition-specified routers. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17390–17401, Stroudsburg, PA, USA. Association for Computational Linguistics… find
  15. Xinyue Liu, Zeyang Qin, Zeyu Wang, Wenxin Liang, Linlin Zong, and Bo Xu. 2025. Conditional semantic textual similarity via conditional contrastive learning. In Proceedings of the 31st International Conference on Computational Linguistics, pages 4548–4560. find
  16. Goutam Majumder, Partha Pakray, Alexander Gelbukh, and David Pinto. 2016. Semantic textual similarity methods, tools, and applications: A survey. Computación y Sistemas, 20(4):647–665. find
  17. Christopher D Manning, Prabhakar Raghavan, and Hinrich Schutze. 2008. Introduction to Information Retrieval. Cambridge University Press. find
  18. Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. MTEB: Massive text embedding benchmark. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2014–2037, Dubrovnik, Croatia. Association for Computational Linguistic…
  19. Sara Rajaee and Mohammad Taher Pilehvar. 2022. An isotropy analysis in the multilingual BERT embedding space. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1309–1316, Dublin, Ireland. Association for Computational Linguistics. find
  20. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3980–3990,…
  21. Julian Risch, Timo Möller, Julian Gutsch, and Malte Pietsch. 2021. Semantic answer similarity for evaluating question answering models. In Proceedings of the 3rd Workshop on Machine Reading for Question Answering, pages 149–157, Stroudsburg, PA, USA. Association for Computational Linguistics. find
  22. Jianlin Su, Jiarun Cao, Weijie Liu, and Yangyiwen Ou. 2021. Whitening sentence representations for better semantics and faster retrieval. Preprint, arXiv:2103.15316. find
  23. Chongyang Tao, Tao Shen, Shen Gao, Junshuo Zhang, Zhen Li, Kai Hua, Wenpeng Hu, Zhengwei Tao, and Shuai Ma. 2025. Llms are also effective embedding models: An in-depth overview. Preprint, arXiv:2412.12591. — [arXiv:2412.12591](https://arxiv.org/abs/2412.12591) find
  24. Jingxuan Tu, Keer Xu, Liulu Yue, Bingyang Ye, Kyeongmin Rim, and James Pustejovsky. 2024. Linguistically conditioned semantic textual similarity. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1161–1172, Stroudsburg, PA, USA.… find
  25. Kosuke Yamada and Peinan Zhang. 2025. Out-of-the-box conditional text embeddings from large language models. Preprint, arXiv:2504.16411. — [arXiv:2504.16411](https://arxiv.org/abs/2504.16411) find
  26. Young Yoo, Jii Cha, Changhyeon Kim, and Taeuk Kim. 2024. Hyper-CL: Conditioning sentence representations with hypernetworks. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 700–711, Bangkok, Thailand. Association for Computati… find
  27. Gaifan Zhang, Yi Zhou, and Danushka Bollegala. 2025. Annotating training data for conditional semantic textual similarity measurement using large language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 27015–27027, Suzhou, China. Association… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-case-condition-aware-sentence-embeddings/