Elamur Research⌕
☰
home / papers / 2026 / Team UBSE at SemEval-2026 Task 4: Adapting Generalist Embeddings for Narrative Representations

Team UBSE at SemEval-2026 Task 4: Adapting Generalist Embeddings for Narrative Representations

Paper · 2026 · Marius Marogel and Marius Popescu

Summary

This paper describes the UBSE system for SemEval-2026 Task 4. The task asks how narratively similar stories are, judged by abstract theme, course of action and outcome. Track A picks which of two candidates is closer to an anchor story. Track B produces one vector per story.

The authors build a single story embedding and use it for both tracks:

  • GPT-4o-mini rewrites each story into the three aspects.
  • Instruction-prompted generalist encoders (Qwen3-Embedding-0.6B and QZhou-Embedding) embed the full story and each aspect.
  • These embeddings are combined with weights derived from the Track A dev triplets.
  • The result is debiased by removing the mean-direction projection and the top-k singular directions.
  • Finally, a non-negative diagonal Mahalanobis-like scaling is fitted with ridge-regularized NNLS.

Every stage improves dev accuracy. On the test set, however, post-processing hurt Qwen3 and the learned scaling overfit. The system placed 12th of 27 in Track B, above the StoryEmb baseline, but fell below the GPT-4o-mini baseline in Track A. The main lesson is that common embedding post-processing tricks depend on the encoder and should not be applied by default.

Key points

  • Leaderboard: 12th of 27 in Track B and 26th of 44 in Track A.
  • Dev gains with Qwen3-Embedding-0.6B: Track A accuracy rose from 0.59 (no instructions) to 0.85 with the full pipeline, and Track B rose from 0.63 to 0.78. QZhou-Embedding went from 0.54 to 0.73 (A) and from 0.55 to 0.68 (B).
  • Test results on Track B (Qwen3): aspect weighting gave only a small gain (0.63 → 0.64). Debiasing then reduced accuracy to 0.62, while QZhou benefited from post-processing consistently.
  • Aspect weights: Qwen3 weights the aspects almost evenly, with the whole story highest (0.282). QZhou favours the abstract theme (0.397) over the course of action (0.165).
  • Hyperparameters: k = 2 singular vectors were removed for Qwen3 and k = 1 for QZhou. The NNLS regularization was α = 0.1, implemented with SciPy and PyTorch SVD.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of BucharestBucharest · Romania21 works · 516 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

Cite

Cite as Marius Marogel and Marius Popescu. 2026. Team UBSE at SemEval-2026 Task 4: Adapting Generalist Embeddings for Narrative Representations. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.258

@inproceedings{marogel2026team,
  title = {{Team UBSE at SemEval-2026 Task 4: Adapting Generalist Embeddings for Narrative Representations}},
  author = {Marius Marogel and Marius Popescu},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.258},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.258}
}

References

22 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Mousumi Akter and Shubhra Kanti Karmaker Santu. 2023. Fans: A facet-based narrative similarity metric. arXiv preprint arXiv:2309.04823. — [arXiv:2309.04823](https://arxiv.org/abs/2309.04823) find
  2. Sanjeev Arora, Yingyu Liang, and Tengyu Ma. 2017. A simple but tough-to-beat baseline for sentence embeddings. In International conference on learning representations. find
  3. Hongliu Cao. 2024. Recent advances in text embedding: A comprehensive review of top-performing methods on the mteb benchmark. Preprint, arXiv:2406.01607. — [arXiv:2406.01607](https://arxiv.org/abs/2406.01607) find
  4. Courtney Corley and Rada Mihalcea. 2005. Measuring the semantic similarity of texts. In Proceedings of the ACL Workshop on Empirical Modeling of Semantic Equivalence and Entailment, pages 13–18, Ann Arbor, Michigan. Association for Computational Linguistics. and William El Sayed. 2023. Mistral 7b. P…
  5. O-Joun Lee and Jason J Jung. 2020. Story embedding: Learning distributed representations of stories based on character networks. Artificial Intelligence, 281:103235. find
  6. Chaofan Li, MingHao Qin, Shitao Xiao, Jianlyu Chen, Kun Luo, Yingxia Shao, Defu Lian, and Zheng Liu. 2024. Making text embedders few-shot learners. Preprint, arXiv:2409.15700. find
  7. Haoyu Liang, Youran Sun, Yunfeng Cai, Jun Zhu, and Bo Zhang. 2025. Jailbreaking llms’ safeguard with universal magic words for text embedding models. Preprint, arXiv:2501.18280. — [arXiv:2501.18280](https://arxiv.org/abs/2501.18280) find
  8. Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. 2016. A corpus and cloze evaluation for deeper understanding of commonsense stories. In Proceedings of the 2016 Conference of the North American Chapter of the Associati… find
  9. Gene H Golub and Charles F Van Loan. 2013. Matrix computations. JHU press. find
  10. Jiaqi Mu, Suma Bhat, and Pramod Viswanath. 2018. All-but-the-top: Simple and effective post-processing for word representations. Preprint, arXiv:1702.01417. find
  11. Wael H Gomaa, Aly A Fahmy, and 1 others. 2013. A survey of text similarity approaches. international journal of Computer Applications, 68(13):13–18. find
  12. Niklas Muennighoff, Nouamane Tazi, Loïc Magne, and Nils Reimers. 2023. Mteb: Massive text embedding benchmark. Preprint, arXiv:2210.07316.
  13. Hans Ole Hatzel, Ekaterina Artemova, Haimo Stiemer, Evelyn Gius, and Chris Biemann. 2026. SemEval-2026 Task 4: Narrative similarity and narrative representation learning. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026), San Diego, CA, USA. Association for Comp… find
  14. Jianmo Ni, Gustavo Hernandez Abrego, Noah Constant, Ji Ma, Keith Hall, Daniel Cer, and Yinfei Yang. 2022. Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models. In Findings of the association for computational linguistics: ACL 2022, pages 1864–1874.
  15. Hans Ole Hatzel and Chris Biemann. 2024a. Story embeddings—narrative-focused representations of fictional stories. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 5931–5943. find
  16. Hans Ole Hatzel and Chris Biemann. 2024b. Tell me again! a large-scale dataset of multiple summaries for the same story. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 15732–15741, Torino, Italia. EL… find
  17. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… — GPT-4o System Card
  18. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zach DeVito, Martin Raison, Alykhan Te-jani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, and 2 others. 20… — PyTorch: An Imperative Style, High-Performance Deep Learning Library
  19. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. find
  20. Xingyu Ren, Youran Sun, and Haoyu Liang. 2025. Correcting mean bias in text embeddings: A refined renormalization with training-free improvements on mmteb. Preprint, arXiv:2511.11041. — [arXiv:2511.11041](https://arxiv.org/abs/2511.11041) find
  21. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. CoRR, abs/1706.03762.
  22. Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Cournapeau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nelson, Eric Jones, Robert Kern, Er… — SciPy 1.0 — Fundamental Algorithms for Scientific Computing in Python

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/marogel-2026-team-ubse-semeval-2026-task/