blue at SemEval-2026 Task 4: Synergizing Long-Context Reranking with Semantic Similarity for Narrative Alignment
Paper · 2026 · Krish Sharma, Lakksh Sharma, Rhea Singhal et al.
Summary
Team blue tackled Track A (Pairwise Similarity) of SemEval-2026 Task 4. Given an anchor story and two candidates, a system must pick the candidate closer in abstract theme, course of action and outcome. The authors argue that 512-token encoders such as BERT and RoBERTa often cut off the endings that carry resolutions and twists, which leaves only shallow lexical comparison. They built a two-stream weighted ensemble:
- Structural stream: Jina Reranker v2-base-multilingual acts as a cross-encoder over overlapping 1,024-token windows, with max-pooling of chunk scores.
- Semantic stream: a RoBERTa-Large model (cross-encoder/stsb-roberta-large) produces mean-pooled embeddings compared by cosine similarity. It is meant to catch tonal and stylistic mismatches.
The streams are fused with weight α = 0.7 on the structural score. The system used only the official English data. It reached Pearson r = 0.63, and the development-set ablations show gains as reachable context grows. The paper does not report the official primary metric, accuracy. The work suggests that access to story endings drives performance in narrative similarity.
Key points
- Development-set ablation (Pearson r):
- RoBERTa-Large at 512 tokens: 0.51
- BGE-Large cross-encoder at 512 tokens: 0.56
- Jina Reranker v2 with chunking: 0.59
- Hybrid ensemble: 0.63, which is also the value the paper reports on the official test set
- Fusion weight: a linear combination with α = 0.7 for the structural score, found by grid search on dev. The authors read this as evidence that long-context structure dominates.
- Setup:
- RoBERTa fine-tuned for 3 epochs, batch 16, LR 2×10⁻⁵ with AdamW, inputs capped at 512 tokens
- Jina run in FP16 inference
- Code: github.com/Demonn1567/SemEval_Task4
- Error analysis (50 sampled errors):
- Purely allegorical parallels (e.g., wolfpack vs. boardroom) are missed.
- A "satire trap" occurs when an earnest narrative and a mocking version of it follow the same underlying plot.
- Limitations:
- Chunks get no cross-chunk attention.
- α may not transfer to other domains.
- Cross-lingual ability is untested, since RoBERTa is English-only.
- Track B was not attempted.
- BGE-M3 was explored but left out of the final system.
Institutions
IndiaFields, topics and concepts
📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksComputer Vision and Pattern Recognition · 2,869 works
🧭 Topics 1Research subjects and languages the work is about.Information retrieval & search · 473 works
Cite
Cite as Krish Sharma, Lakksh Sharma, Rhea Singhal and Jatin Bedi. 2026. blue at SemEval-2026 Task 4: Synergizing Long-Context Reranking with Semantic Similarity for Narrative Alignment. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).
@inproceedings{sharma2026blue,
title = {{blue at SemEval-2026 Task 4: Synergizing Long-Context Reranking with Semantic Similarity for Narrative Alignment}},
author = {Krish Sharma, Lakksh Sharma, Rhea Singhal and Jatin Bedi},
booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
year = {2026},
doi = {10.18653/v1/2026.semeval-1.305},
url = {https://doi.org/10.18653/v1/2026.semeval-1.305}
}References
- Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150.
- Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. BGE M3-embedding: find
Updated
This page in Markdown → https://md.research.elamur.ai/free/en/papers/sharma-2026-blue-semeval-2026-task-4/