Elamur Research⌕
☰
home / papers / 2026 / One Sentence, Two Embeddings: Contrastive Learning of Explicit and Implicit Semantic Representations

One Sentence, Two Embeddings: Contrastive Learning of Explicit and Implicit Semantic Representations

Paper · 2026 · Kohei Oda, Po-Min Chuang, Kiyoaki Shirai et al.

Summary

Conventional sentence encoders produce one vector per sentence, so they tend to miss non-literal meaning that comes from figurative or pragmatic usage. The authors propose DualCSE, which gives each sentence two embeddings in a common space: one for its literal (explicit) meaning and one for its implied (implicit) meaning. Training uses the INLI dataset, where each premise has four hypotheses: explicit-entailment, implied-entailment, neutral and contradiction. A new contrastive loss extends SimCSE in two ways:

  • Between sentences: it pulls each premise toward its matching entailment hypothesis and away from the contradiction.
  • Within a sentence: it keeps a hypothesis's two vectors close but separates the two vectors of the more ambiguous premise.

Two RoBERTa-based variants are built: a cross-encoder conditioned on an "explicit"/"implicit" tag, and a bi-encoder made of two separate models. In entailment recognition, DualCSE slightly beats a SimCSE model trained on the same data. For ranking sentences by implicitness, the gap between a sentence's two vectors gives a score competitive with a specialised model. The approach lets users choose which meaning to use for retrieval or classification.

Key points

  • RTE on INLI (average accuracy):
    • DualCSE-Bi 80.38% and DualCSE-Cross 80.18%.
    • SimCSE (INLI) 79.40% and SimCSE (SNLI+MNLI) 67.68%.
    • On implied-entailment items, the cross-encoder reaches 73.40%, versus 69.10% for SimCSE (INLI).
    • Gemini-1.5-Pro (few-shot) reaches 91.40%.
  • EIS (implicitness score = 1 − cosine of the two vectors):
    • In-domain on INLI, accuracy is near-perfect (99.97–100%), but a simple LENGTH baseline also reaches 99.90%.
    • On Wang et al.'s out-of-domain data, DualCSE scores 77–79%, close to ImpScore (INLI) at 81.56%.
  • Ablation: removing the contradiction terms lowers RTE accuracy to about 64.6%. Removing the intra-sentence terms lowers EIS on INLI to 92.25%. Removing both lowers EIS to 32.75%.
  • Setup: RoBERTa base, temperature τ = 0.05. Code is released at github.com/iehok/DualCSE.
  • Limitations: training used only INLI, which has limited variety. Extensions to LLM-based encoders and broader pragmatics are left for future work.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 works

Cite

Cite as Kohei Oda, Po-Min Chuang, Kiyoaki Shirai and Natthawut Kertkeidkachorn. 2026. One Sentence, Two Embeddings: Contrastive Learning of Explicit and Implicit Semantic Representations. Findings of the Association for Computational Linguistics: EACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-eacl.74

@inproceedings{oda2026one,
  title = {{One Sentence, Two Embeddings: Contrastive Learning of Explicit and Implicit Semantic Representations}},
  author = {Kohei Oda, Po-Min Chuang, Kiyoaki Shirai and Natthawut Kertkeidkachorn},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-eacl.74},
  url = {https://doi.org/10.18653/v1/2026.findings-eacl.74}
}

References

28 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. 2024. LLM2vec: Large language models are secretly powerful text encoders. In First Conference on Language Modeling.
  2. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642, Lisbon, Portugal. Association for Computati…
  3. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 1597–1607. PMLR.
  4. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
  5. Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. SimCSE: Simple contrastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894–6910, Online and Punta Cana, Dominican Republic. Association for Computational Linguistic…
  6. Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. 2022. ToxiGen: A large-scale machine-generated dataset for adversarial and implicit hate speech detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1:…
  7. Shreya Havaldar, Hamidreza Alvari, John Palowitch, Mohammad Javad Hosseini, Senaka Buthpitiya, and Alex Fabrikant. 2025. Entailed between the lines: Incorporating implication into NLI. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),…
  8. Ting Jiang, Shaohan Huang, Zhongzhi Luan, Deqing Wang, and Fuzhen Zhuang. 2024. Scaling sentence embeddings with large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 3182–3196, Miami, Florida, USA. Association for Computational Linguistics.
  9. Ting Jiang, Jian Jiao, Shaohan Huang, Zihan Zhang, Deqing Wang, Fuzhen Zhuang, Furu Wei, Haizhen Huang, Denvy Deng, and Qi Zhang. 2022. Prompt-BERT: Improving BERT sentence embeddings with prompts. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8826–… find
  10. Xianming Li and Jing Li. 2024. AoE: Angle-optimized embeddings for semantic textual similarity. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1825–1839, Bangkok, Thailand. Association for Computational Linguistics. find
  11. Xianming LI, Zongxi Li, Jing Li, Haoran Xie, and Qing Li. 2025. ESE: Espresso sentence embeddings. In The Thirteenth International Conference on Learning Representations. find
  12. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. Preprint, arXiv:1907.11692.
  13. Yuxin Wang, Xiaomeng Zhu, Weimin Lyu, Saeed Hassanpour, and Soroush Vosoughi. 2025. Impscore: A learnable metric for quantifying the implicitness level of sentences. In The Thirteenth International Conference on Learning Representations. find
  14. Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142–150, Portland, Oregon…
  15. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pa…
  16. Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. MTEB: Massive text embedding benchmark. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2014–2037, Dubrovnik, Croatia. Association for Computational Linguistic…
  17. Jianmo Ni, Gustavo Hernandez Abrego, Noah Constant, Ji Ma, Keith Hall, Daniel Cer, and Yinfei Yang. 2022. Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1864–1874, Dublin, Ireland. Associatio…
  18. Maria Pontiki, Dimitris Galanis, John Pavlopoulos, Harris Papageorgiou, Ion Androutsopoulos, and Suresh Manandhar. 2014. SemEval-2014 task 4: Aspect based sentiment analysis. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 27–35, Dublin, Ireland. Associa…
  19. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…
  20. Kosuke Yamada and Peinan Zhang. 2025. Out-of-the-box conditional text embeddings from large language models. Preprint, arXiv:2504.16411. — [arXiv:2504.16411](https://arxiv.org/abs/2504.16411) find
  21. Shohei Yoda, Hayato Tsukagoshi, Ryohei Sasano, and Koichi Takeda. 2024. Sentence representations via Gaussian embedding. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers), pages 418–425, St. Julian’s, Malta. Associ… find
  22. Dun Zhang, Jiacheng Li, Ziyang Zeng, and Fulong Wang. 2025. Jasper and stella: distillation of sota embedding models. Preprint, arXiv:2412.19048. — [arXiv:2412.19048](https://arxiv.org/abs/2412.19048) find
  23. Xin Zhang, Yanzhao Zhang, Dingkun Long, Wen Xie, Ziqi Dai, Jialong Tang, Huan Lin, Baosong Yang, Pengjun Xie, Fei Huang, Meishan Zhang, Wenjie Li, and Min Zhang. 2024. mGTE: Generalized long-context text representation and reranking models for multilingual text retrieval. In Proceedings of the 2024…
  24. A Table 6 shows the statistics of the INLI dataset and Wang’s dataset (Wang et al., 2025). find
  25. Yiqun Sun, Qiang Huang, Anthony K. H. Tung, and Jun Yu. 2025. Text embeddings should capture implicit semantics, not just surface meaning. Preprint, arXiv:2506.08354. — [arXiv:2506.08354](https://arxiv.org/abs/2506.08354) find
  26. Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). find
  27. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2024. Text embeddings by weakly-supervised contrastive pre-training. Preprint, arXiv:2212.03533.
  28. Dataset Statistics Dataset train development test INLI Wang et al. (2025) 32,000 101,320 4,000 5,630 4,000 5,630 *Table 6: Statistics of datasets. This table shows the number of premise-hypothesis pairs for the INLI dataset and that of implicit-explicit sentence pairs for Wang’s dataset.* find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/oda-2026-one-sentence-two-embeddings-contrastive/