Elamur Research⌕
☰
home / papers / 2026 / CITD@UIT at SemEval-2026 Task 4: Structured Reasoning and Metric Specialization for Narrative Similarity

CITD@UIT at SemEval-2026 Task 4: Structured Reasoning and Metric Specialization for Narrative Similarity

Paper · 2026 · Thach Nguyen, Duc-Vu Nguyen and Dang Thin

Summary

This paper describes the CITD@UIT entry to SemEval-2026 Task 4, which asks whether systems can judge how similar stories are in narrative terms rather than in wording. The task has two tracks. Track A asks which of two candidate stories is closer to an anchor. Track B asks for story embeddings whose distances reflect narrative similarity.

For Track A, the authors tested three LLM-based designs in turn, each built to address the previous one's weaknesses. The best was a two-step Gemini pipeline. Gemini 2.5 Flash first rewrites each story into a cleaner, safety-compliant form. Gemini 2.5 Pro then compares the candidates with chain-of-thought prompting in both orders, focusing on moral agency and outcomes.

For Track B, which ships only unlabeled stories, they fine-tuned BGE-M3 with LoRA. Part of the training data was synthetic adversarial examples modeled on Track A errors.

The results suggest that outcome and the protagonist's moral path matter more than shared vocabulary. They also show that relative ranking suits LLMs better than absolute scoring.

Key points

  • Track A approaches, in order:
    • A DeBERTa-v3-base cross-encoder with a Gemini 2.5 Flash/Pro arbitration ensemble (66.5% dev).
    • DSPy decomposition into five narrative dimensions with absolute 0–100 scoring (68.0% dev, 66.75% test).
    • Symmetric pairwise CoT (77.39% dev, 69.25% test), which ranked 17th of 46 teams.
  • Track B used BGE-M3 (LoRA), trained with Multiple Negatives Ranking Loss, an instruction prefix, CLS pooling, and L2-normalized 1024-dimensional vectors. It scored 68.75% and ranked 6th of 26 teams.
  • The Track B training set had 3,223 samples. These combined Track A dev data, the organizers' synthetic data, and 1,126 new adversarial "trap" examples.
  • Applying the Track B embeddings directly to the Track A test set gave 59.00%, about 10 points below the reasoning pipeline.
  • The main error types were:
    • Lexical Gravity: shared motifs mislead the model.
    • Moral agency rigidity: the model handles ethically ambiguous characters poorly.
    • Thematic over-abstraction: the sanitization step removes distinctive details.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Vietnam National University Ho Chi Minh CityHo Chi Minh City · Vietnam21 works · 14 citations🎓University of Information TechnologyHo Chi Minh City · Vietnam13 works · 7 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 1Research subjects and languages the work is about.Reasoning · 1,685 works

Cite

Cite as Thach Nguyen, Duc-Vu Nguyen and Dang Thin. 2026. CITD@UIT at SemEval-2026 Task 4: Structured Reasoning and Metric Specialization for Narrative Similarity. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.semeval-1.158

@inproceedings{nguyen2026citd,
  title = {{CITD@UIT at SemEval-2026 Task 4: Structured Reasoning and Metric Specialization for Narrative Similarity}},
  author = {Thach Nguyen, Duc-Vu Nguyen and Dang Thin},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.158},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.158}
}

References

7 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Nathanael Chambers and Dan Jurafsky. 2008. Unsupervised learning of narrative event chains. In Proceedings of ACL-08: HLT, pages 789–797, Columbus, Ohio. Association for Computational Linguistics. find
  2. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. Preprint, arXiv:2402.03216.
  3. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, and 1 others. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. Preprint, arXiv:2502.10001.
  4. Mark Alan Finlayson. 2012. Learning Narrative Structure from Annotated Folktales. PhD thesis, Massachusetts Institute of Technology, Cambridge, MA. Advisor: Patrick H. Winston. find
  5. Hans Ole Hatzel, Ekaterina Artemova, Haimo Stiemer, Evelyn Gius, and Chris Biemann. 2026. SemEval-2025 task 4: Narrative similarity and narrative representation learning. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026), San Diego, CA, USA. find
  6. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2021. Debertav3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing. Preprint, arXiv:2111.09543.
  7. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/nguyen-2026-citd-uit-semeval-2026-task/