📄 Exploring Coreference Resolution in Glosses of German Sign Language

Summary

The paper investigates coreference resolution in German Sign Language (DGS/GSL) at the gloss level. Glosses are linear textual labels that specialists use to transcribe signs. The authors first analyze how glosses refer to entities. These include noun phrases, among them iconic and compound signs, as well as $INDEX pointing signs anchored to locations in the signing space. Frequent ellipsis and verb agreement, which is not visible in the glosses themselves, are considered separately. The authors then manually annotated entity coreference in the glosses of three videos from the Public DGS Corpus. The earlier DGS-Coref dataset contained only pronominal links, whereas the new annotation covers all mention types. On these data, GPT-4o and DeepSeek-v3 were evaluated in zero-shot and few-shot settings. The best result, a CoNLL F1 of 68.83, was achieved by GPT-4o with an example in the prompt, so both models performed only moderately. The work is important as an early step toward automatic entity tracking in sign languages. It also shows what information is lost when video is replaced with glosses.

Key points

• The annotation covers three videos with a total duration of 990 s: 336 mentions and 89 entities; singleton entities were not annotated. Inter-annotator agreement between the two annotators, measured by Krippendorff's α, is 0.93. The data are openly released in JSON format in the GitHub repository Coref_GLS_GSL.
• Variants of the same gloss with different numeric suffixes, for example ICH1 and ICH2, were treated as a single entity. Ambiguous cases were excluded. Deaf sign language users were not involved in the annotation.
• Experimental setup: the data were split into 20 prompts by topic, with temperature 0. In the few-shot setting, one similar example with gold annotation was added to the prompt. Evaluation used MUC, B³, CEAFe and CoNLL F1 with macro-averaging.
• GPT-4o in the few-shot setting scored 68.83 CoNLL F1. This is about 6 points higher than its zero-shot version and 7 points higher than DeepSeek-v3 few-shot (61.49). DeepSeek-v3 shows high recall (73.46) but low precision (52.91).
• GPT-4o performs better on dialogues than on monologues: it correctly links one interlocutor's ICH with the other's DU. DeepSeek-v3 is weaker on dialogues but makes more willing use of the cue from the provided example.