Elamur Research⌕
☰
home / papers / 2026 / Spinfo Cologne at SemEval-2026 Task 4: Explainable Creation of Narrativity Embeddings

Spinfo Cologne at SemEval-2026 Task 4: Explainable Creation of Narrativity Embeddings

Paper · 2026 · Janis Pagel and Nils Reiter

Summary

SemEval-2026 Task 4 asks systems to judge narrative similarity between story summaries. In Track A, a system chooses which of two candidates is narratively closer to an anchor text. In Track B, it outputs a vector for each story. The Cologne team prioritized interpretability. Instead of encoding whole texts end-to-end, they prompt openai/gpt-4o-mini to extract the three aspects defined by the task: themes, course of action, and outcome, each under strict length limits. Each aspect is embedded either by averaging GloVe vectors or with the all-MiniLM-L6-v2 sentence transformer. The aspect vectors are then concatenated, and cosine similarity decides Track A. The system was developed on 200 triples of Wikipedia plot summaries of books and films. The sentence-transformer variant clearly beats GloVe, which stays at chance level. The official submission scored 60.25% on Track A and 57.75% on Track B. Ablations show only a small gain over embedding raw text and unequal usefulness of the aspects. The value of the work lies in its transparent intermediate outputs, which reveal failure causes such as theme overlap masking differences between events.

Key points

  • Prompt constraints: the model may list no more than five single-word themes, no more than five brief plot points capped at 10 words each, and one outcome limited to 10 words. The authors report that the model often ignored the instruction to avoid proper names.
  • Development results for Track A: SBERT (all-MiniLM-L6-v2) reaches 0.6150 accuracy and 0.6244 F1. GloVe (300d, 6B tokens) reaches 0.49, against random at about 0.50 and majority at 0.505. The majority baseline has a higher F1 (0.6711).
  • In the ablation runs, the aspect pipeline scores 0.59 accuracy versus 0.56 for raw-text MiniLM embeddings.
  • Dropping themes gives the best ablation accuracy (0.60), even though themes alone perform well (0.5949). Hand-picked weightings also peak at 0.60.
  • Every aspect gets an equal 384-dimensional slot regardless of how informative it is. The authors suggest learned weights or attention over aspects, stronger encoders, and better prompting, such as few-shot examples or validation steps.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of CologneCologne · Germany9 works · 407 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 worksLiterature and Literary Theory · 519 works

Cite

Cite as Janis Pagel and Nils Reiter. 2026. Spinfo Cologne at SemEval-2026 Task 4: Explainable Creation of Narrativity Embeddings. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.299

@inproceedings{pagel2026spinfo,
  title = {{Spinfo Cologne at SemEval-2026 Task 4: Explainable Creation of Narrativity Embeddings}},
  author = {Janis Pagel and Nils Reiter},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.299},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.299}
}

References

13 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. H. Porter Abbott. 2014. Narrativity. In: the living handbook of narratology. find
  2. Hans Ole Hatzel, Ekaterina Artemova, Haimo Stiemer, Natalia Fedorova, Evelyn Gius, and Chris Biemann. 2025. Narrative similarity – annotation guidelines. find
  3. Hans Ole Hatzel, Ekaterina Artemova, Haimo Stiemer, Evelyn Gius, and Chris Biemann. 2026. SemEval-2026 Task 4: Narrative Similarity and Narrative Representation Learning. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026), San Diego, CA, USA. find
  4. Hans Ole Hatzel and Chris Biemann. 2023. Narrative cloze as a training objective: Towards modeling stories using narrative chain embeddings. In Proceedings of the 5th Workshop on Narrative Understanding, pages 118–127, Toronto, Canada. Association for Computational Linguistics. find
  5. Hans Ole Hatzel and Chris Biemann. 2024a. Story embeddings — narrative-focused representations of fictional stories. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 5931–5943, Miami, Florida, USA. Association for Computational Linguistics. find
  6. Hans Ole Hatzel and Chris Biemann. 2024b. Tell me again! a large-scale dataset of multiple summaries for the same story. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 15732–15741, Torino, Italia. EL… find
  7. Hans Ole Hatzel, Fynn Petersen-Frey, Tim Fischer, and Chris Biemann. 2023. Dimensions of Similarity: Towards Interpretable Dimension-Based Text Similarity. In Kobi Gal, Ann Nowé, Grzegorz J. Nalepa, Roy Fairstein, and Roxana Rădulescu, editors, Frontiers in Artificial Intelligence and Applications.… find
  8. Juri Opitz, Lucas Moeller, Andrianos Michail, Sebastian Padó, and Simon Clematide. 2025. Interpretable text embeddings and text similarity explanation: A survey. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 22303–22319, Suzhou, China. Association f… find
  9. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar. Association for Computational Linguistics.
  10. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…
  11. Shashank Srivastava and Nebojsa Jojic. 2018. A spatial model for extracting and visualizing latent discourse structure in text. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2268–2277, Melbourne, Australia. Association for C… find
  12. Igor Sterner, Alex Lascarides, and Frank Keller. 2026. Contrastive learning with narrative twins for modeling story salience. Preprint, arXiv:2601.07765.
  13. Sean Wilner, Daniel Woolridge, and Madeleine Glick. 2021. Narrative embedding: Re-Contextualization through attention. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1393–1405, Online and Punta Cana, Dominican Republic. Association for Computational… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/pagel-2026-spinfo-cologne-semeval-2026-task/