Elamur Research⌕
☰
home / papers / 2026 / Supervised Contrastive Fine-Tuning for Active Few-Shot Learning

Supervised Contrastive Fine-Tuning for Active Few-Shot Learning

Summary

Research on Active Few-Shot Learning (AFSL) has mostly improved query strategies, which decide what to annotate. The training objective used after each annotation round is usually plain cross-entropy (CE). The authors argue that CE leaves the embedding space poorly structured when each round adds only a few labels, so annotation effort is wasted.

They propose SCL-HNM, a hybrid loss for text classification. CE remains the main term, and a supervised contrastive regularizer with hard negative mining is added. The loss acts directly on mean-pooled encoder embeddings of google/flan-t5-base, with no projection head. To isolate the objective from data selection, the query strategy is fixed to Rep(En)-ClUn(En). The model is also reset to pre-trained weights before training at each support-set size.

On five benchmark datasets, SCL-HNM consistently beats CE baselines. A supplementary test with a different backbone and acquisition function shows gains as well. The work suggests that better learning objectives complement better sample selection. Code and data splits are released.

Key points

  • Method: CE combined with a supervised contrastive term (after Khosla et al.) plus hard negative mining, in the spirit of SCHaNe.

  • Projection head: deliberately removed, trading possible disentanglement for fewer parameters and less overfitting.

  • Evaluation protocol: support sets of K = 10, 20, …, 100 with a strict weight reset to θ₀ at each checkpoint. Main experiments use a fixed query strategy, Rep(En)-ClUn(En) (Ahmadnia et al., 2025).

  • Datasets: the language-resource references cite AG's news corpus, the Multilingual Amazon Reviews Corpus and the MPQA Corpus (MPQA-Polarity is used). The full list of five datasets and the main result tables fall in the omitted part of the text.

  • Robustness check: with roberta-base and entropy sampling on MPQA-Polarity (Micro-F1, 5 seeds), SCL-HNM vs. CE scored:

    • K=20: 69.4 vs 60.0
    • K=50: 76.6 vs 74.9
    • K=100: 79.1 vs 78.7

    Standard deviations were lower at every K.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/5p4u2sjsmrcm

@inproceedings{zhang2026supervised,
  title = {{Supervised Contrastive Fine-Tuning for Active Few-Shot Learning}},
  author = {Zhang, Zirui and Ge, Lei and Qiao, Shengyu},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/5p4u2sjsmrcm},
  url = {https://doi.org/10.63317/5p4u2sjsmrcm}
}

References

27 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. S. AHMADNIA, A. Y. JORDEHI, M. H. K. HEYRAN, et al. 2025. Active few-shot learning for text classification. ArXiv. find
  2. S. AHMADNIA, A. YOUSEFI JORDEHI, M. HOS-SEINI KHASHEH HEYRAN, et al. 2024. Opinion mining using pre-trained large language models: Identifying the type, polarity, intensity, expression, and source of private states. In Proceedings of the 2024 Joint International Conference on Computational Linguisti… find
  3. S. BAIK, J. CHOI, H. KIM, et al. 2021. Meta-learning with task-adaptive loss function for few-shot learning. ArXiv. find
  4. T. B. BROWN, B. MANN, N. RYDER, et al. 2020. Language models are few-shot learners. ArXiv. find
  5. E. CHANG, X. SHEN, H. S. YEH, et al. 2021. On training instance selection for few-shot neural text generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers… find
  6. Y. CHEN, Y. ZHANG, C. ZHANG, et al. 2021. Revisiting self-training for few-shot learning of language model. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9125–9135, Online and Punta Cana, Dominican Republic. Association for Computational Lin… find
  7. A. EDWARDS and J. CAMACHO-COLLADOS. 2024. Language models for text classification: Is in-context learning enough? ArXiv. find
  8. L. EIN-DOR, A. HALFON, A. GERA, et al. 2020. Active learning for bert: An empirical study. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7949–7962, Online. Association for Computational Linguistics. find
  9. T. GAO, A. FISCH, and D. CHEN. 2021a. Making pre-trained language models better few-shot learners. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3816… find our card
  10. T. GAO, X. YAO, and D. CHEN. 2021b. Simcse: Simple contrastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6894–6910, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. find our card
  11. B. GUNEL, J. DU, A. CONNEAU, et al. 2021. Supervised contrastive learning for pre-trained language model fine-tuning. ArXiv. find our card
  12. K. HE, H. FAN, Y. WU, et al. 2020. Momentum contrast for unsupervised visual representation learning. ArXiv. find our card
  13. R. KARIMI MAHABADI, L. ZETTLEMOYER, J. HENDERSON, et al. 2022. Prompt-free and efficient few-shot learning with language models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3638–3652, Dublin, Ireland. Association for Compu… find
  14. P. KHOSLA, P. TETERWAK, C. WANG, et al. 2021. Supervised contrastive learning. ArXiv. find
  15. A. LI, W. HUANG, X. LAN, et al. 2020. Boosting few-shot learning with adaptive margin loss. ArXiv. find
  16. X. V. LIN, T. MIHAYLOV, M. ARTETXE, et al. 2022. Few-shot learning with multilingual language models. ArXiv. find our card
  17. Lajanugen LOGESWARAN and Honglak LEE. 2018. An efficient framework for learning sentence representations. In arXiv preprint arXiv:1803.02893. Accessed: 2025-09-17. find our card
  18. Z. LONG, G. KILLICK, R. MCCREADIE, et al. 2023. When hard negative sampling meets supervised contrastive learning. ArXiv. find
  19. T. MÜLLER, G. PÉREZ-TORRÓ, A. BASILE, et al. 2022. Active few-shot learning with fasl. ArXiv. find
  20. B. SETTLES. 2009. Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin-Madison. find our card
  21. T. WANG and P. ISOLA. 2020. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In Proceedings of the 37th International Conference on Machine Learning (ICML), pages 9929–9939. PMLR. find
  22. Y. WANG, Q. YAO, J. KWOK, et al. 2020. Generalizing from a few examples: A survey on few-shot learning. ArXiv. find
  23. T. YIN, N. LIU, and H. SUN. 2021. Self-paced active learning for deep cnns via effective loss function. Neurocomputing, 424:1–8. find
  24. M. YUAN, H. T. LIN, and J. BOYD-GRABER. 2020. Cold-start active learning through self-supervised language modeling. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7935–7948, Online. Association for Computational Linguistics. find
  25. X. ZHAN, Q. WANG, K. HUANG, et al. 2022. A comparative survey of deep active learning. ArXiv. 8. find
  26. Language Resource References 2005. AG’s corpus of news articles. 2020. The Multilingual Amazon Reviews Corpus. find
  27. WIEBE, J. and WILSON, T. and CARDIE, C. and others. 2005. Annotating Expressions of Opinions and Emotions in Language (MPQA Corpus). find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/zhang-2026-supervised-contrastive-fine-tuning-active/