📄 Team Habib Disambiguators at SemEval-2026 Task 5: Assessing Semantic Plausibility using Regularized Transformer Fine-Tuning

Summary

The paper describes a SemEval-2026 Task 5 entry. The task asks systems to rate, on a 1–5 scale, how plausible a given dictionary sense of a homonym is within a short English story whose ending resolves the ambiguity. Rather than choosing one sense, the authors treat the problem as graded prediction on the AmbiStory data (2280 training and 588 development instances). They join the precontext, target sentence and ending with a question about the homonym and its candidate meaning, then feed this to microsoft/deberta-v3-base. The model predicts the full distribution of the five annotator vote options and is trained with KL-divergence. At inference, the score is the expected value of that distribution. Because the data is small and noisy, they add two regularizers: R-Drop, which enforces consistency between two dropout passes, and layer-wise learning rate decay (LLRD). On the official test set the system reaches a Spearman of 0.56 and Accuracy-within-SD of 0.74. The work shows that soft-label training plus strong regularization helps with subjective, low-resource semantic judgments.

Key points

• Each input is a prompt-style sequence (story [SEP] question about homonym and judged meaning), loosely modeled on GlossBERT. The [CLS] representation feeds a head over 5 vote classes.
• Training settings: 20 epochs, batch size 8, AdamW with a head learning rate of 2×10⁻⁵, LLRD factor γ=0.9 applied across 12 layers, and R-Drop weight α=5.0. Training took about 2 hours on one NVIDIA T4.
• Ablation on the combined metric (mean of Spearman and Acc within SD): baseline 0.60, R-Drop only 0.61, LLRD only 0.63, full model 0.65. The text also cites a figure of 0.72 for the full model, which conflicts with this table.
• On the development set, the best epoch was 16, with Spearman ≈0.527, Acc within SD ≈0.725 and combined score ≈0.626.
• Stated limitations include sensitivity to prompt wording, no explicit modeling of the sentence–ending relation, and hyperparameters chosen heuristically.