Elamur Research⌕
☰
home / papers / 2026 / CUNI at SemEval-2026 Task 4: Multi-Head Narrative Aspect Disentanglement via Entangled Synthetic Dataset

CUNI at SemEval-2026 Task 4: Multi-Head Narrative Aspect Disentanglement via Entangled Synthetic Dataset

Paper · 2026 · Jan Mitka and Jindrich Helcl

Summary

The authors address SemEval-2026 Task 4, Track B, in which a system must produce story embeddings. Given an anchor story, cosine similarity on these embeddings decides which of two candidate stories is narratively closer. In a story, abstract theme, course of action and outcome are mixed together, so the team tries to separate them in embedding space. They add three linear aspect heads on top of Qwen3-Embedding-4B. For training, they build a synthetic corpus of controlled story tuples. Each tuple contains an anchor, a stylistic paraphrase that counts as a positive, and three negatives that each change exactly one core aspect. Fine-tuning uses a multi-head, multi-positive variant of InfoNCE, optionally with CKA regularization between heads. On the development set, accuracy rises clearly over the frozen backbone. On the blind test set the gain only partly holds: the system reaches 64.25% and places 14th of 28 teams. The analysis shows that the heads learn some aspect-specific structure. It also shows that the contrastive training objective and the triplet evaluation are not fully aligned. The released dataset is a reusable resource for work on controllable narrative similarity.

Key points

  • Synthetic dataset:
    • Stories were generated by gpt-oss-120b from discrete aspect configurations.
    • Qwen3-30B-A3B-Thinking-2507-FP8 rated each story for coherence and alignment with its configuration.
    • Of 100,000 anchors, 50,245 five-tuples passed the filter (251,225 stories).
    • The data is published as ufal/NarrativeAspect-EntangledSynthetic.
  • Model:
    • The backbone is Qwen3-Embedding-4B, with the representation taken from the EOS token (2560 dimensions).
    • Three normalized 2560-dimensional heads (theme, course, outcome) are concatenated into the final story embedding.
    • Comparisons can also use only a subset of the heads.
  • Development results (200 triplets):
    • The off-the-shelf backbone reaches 65.00%.
    • The disentangled baseline averages 72.40% with the course+outcome head combination, which was the best combination for every multi-head variant.
    • Ablations with a single head, a frozen backbone, or only course and outcome heads all score lower.
  • Test results (400 triplets on Codabench):
    • The system reaches 64.25%, ranking 14th of 28 teams.
    • Development accuracy peaked at about 2,000 training steps, and further optimization of the loss lowered accuracy.
  • Nearest-centroid analysis:
    • The course head recovers course of action best (70.3%), and the theme head recovers theme at 59.7%.
    • The outcome head recovers its own aspect no better than the other aspects (about 31% for each), which indicates leakage between heads.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Charles UniversityPrague · Czechia133 works · 20,686 citations🎓University of OsloOslo · Norway51 works · 4,155 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works

Cite

Cite as Jan Mitka and Jindrich Helcl. 2026. CUNI at SemEval-2026 Task 4: Multi-Head Narrative Aspect Disentanglement via Entangled Synthetic Dataset. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.274

@inproceedings{mitka2026cuni,
  title = {{CUNI at SemEval-2026 Task 4: Multi-Head Narrative Aspect Disentanglement via Entangled Synthetic Dataset}},
  author = {Jan Mitka and Jindrich Helcl},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.274},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.274}
}

References

11 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Marc-André Carbonneau, Julian Zaidi, Jonathan Boilard, and Ghyslain Gagnon. 2022. Measuring disentanglement: A review of metrics. Preprint, arXiv:2012.09276. — [arXiv:2012.09276](https://arxiv.org/abs/2012.09276) find
  2. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. Preprint, arXiv:1810.04805.
  3. Hans Ole Hatzel, Ekaterina Artemova, Haimo Stiemer, Evelyn Gius, and Chris Biemann. 2026. SemEval-2026 Task 4: Narrative similarity and narrative representation learning. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026), San Diego, CA, USA. Association for Comp… find
  4. Hans Ole Hatzel and Chris Biemann. 2024. Story embeddings – narrative-focused representations of fictional stories. In Proceedings of the 62st Annual Meeting of the Association for Computational Linguistics, Miami, Florida. Association for Computational Linguistics. find
  5. Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. 2017. beta-VAE: Learning basic visual concepts with a constrained variational framework. In International Conference on Learning Representations. find
  6. Ilya Loshchilov and Frank Hutter. 2017. Sgdr: Stochastic gradient descent with warm restarts. Preprint, arXiv:1608.03983. find
  7. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. Preprint, arXiv:1711.05101. find
  8. OpenAI. 2025. gpt-oss-120b & gpt-oss-20b model card. Preprint, arXiv:2508.10925.
  9. Qwen Team. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388.
  10. Aäron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. CoRR, abs/1807.03748.
  11. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. 2025. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mitka-2026-cuni-semeval-2026-task-4/