Elamur Research⌕
☰
home / papers / 2026 / Leveraging Unannotated Sign Language Data via a Robust Data Augmentation Method for Contrastive Representation Learning

Leveraging Unannotated Sign Language Data via a Robust Data Augmentation Method for Contrastive Representation Learning

Paper · 2026 · Ariel Basso Madjoukeng, Pierre Poitier, Belise Edith Kenmogne et al.

Summary

Annotated sign language data is scarce. The paper cites roughly 100 hours of expert work to label a single hour of video, and many of the world's 300+ sign languages are only partially annotated. Contrastive self-supervised pretraining can exploit unlabelled video, but the quality of what it learns depends on how positive pairs are augmented. The authors identify two problems in sign language recognition (SLR). First, generic augmentations such as vertical flips or large rotations do not resemble real signing. Second, a recent specialized method (Madjoukeng et al., 2026) shuffles frames only in non-informative segments, such as repositioning and coarticulation, and leaves the sign-bearing frames untouched. That makes the learned features fragile to signer variability, occlusion and noise. The proposed augmentation does two things to skeleton sequences extracted with MediaPipe: it degrades the irrelevant segments and applies small, realistic perturbations to the relevant ones. In experiments on several datasets, including LSFB and ASL-Citizen, it improves accuracy in linear evaluation and in semi-supervised fine-tuning with only 30% of the labels. This makes it particularly useful for low-resource sign languages.

Key points

  • Two goals for positive pairs: degrade non-relevant segments (repositioning, coarticulation) and simulate subtle variations in the relevant segments, so that representations become more robust to differences between signers.
  • Builds on SSL-SLR (Madjoukeng et al., 2026, TMLR), whose augmentation reorders frames only in non-informative parts of a video.
  • Data: pose skeletons extracted with MediaPipe. Example videos come from LSFB (French Belgian Sign Language, more than 4,567 distinct signs) and ASL-Citizen (2,731 signs).
  • Related methods: the paper reviews the contrastive frameworks SimCLR, MoCo, BYOL, SimSiam and SL-FPN as the family of methods the augmentation is meant to serve.
  • Main result: accuracy improves under linear evaluation and under semi-supervised learning with 30% annotations. Exact figures fall in the portion of the text that was not available.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of NamurNamur · Belgium6 works · 21 citations🔬Namur Digital InstituteNamur · Belgium4 works · 4 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksComputer Vision and Pattern Recognition · 2,869 worksHuman-Computer Interaction · 141 works

Cite

Cite as Ariel Basso Madjoukeng, Pierre Poitier, Belise Edith Kenmogne, Adelaide Couplet, Margaux Leleu and Frenay Benoit. 2026. Leveraging Unannotated Sign Language Data via a Robust Data Augmentation Method for Contrastive Representation Learning. Proceedings of the LREC 2026 12th Workshop on the Representation and Processing of Sign Languages: Language in Motion.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/4qnrmhtbv9cz

@inproceedings{madjoukeng2026leveraging,
  title = {{Leveraging Unannotated Sign Language Data via a Robust Data Augmentation Method for Contrastive Representation Learning}},
  author = {Ariel Basso Madjoukeng, Pierre Poitier, Belise Edith Kenmogne, Adelaide Couplet, Margaux Leleu and Frenay Benoit},
  booktitle = {Proceedings of the LREC 2026 12th Workshop on the Representation and Processing of Sign Languages: Language in Motion},
  year = {2026},
  doi = {10.63317/4qnrmhtbv9cz},
  url = {https://doi.org/10.63317/4qnrmhtbv9cz}
}

References

27 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Nikolas Adaloglou, Theocharis Chatzis, Ilias Papastratis, Andreas Stergioulas, Georgios Th Papadopoulos, Vassia Zacharopoulou, George J Xydopoulos, Klimnis Atzakas, Dimitris Papazachariou, and Petros Daras. 2021. A comprehensive study on deep learning-based methods for sign language recognition. IEE… find
  2. Xinlei Chen and Kaiming He. 2021. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15750– 15758. find
  3. Aashaka Desai, Lauren Berger, Fyodor Minakov, Nessa Milano, Chinmay Singh, Kriston Pumphrey, Richard Ladner, Hal Daumé III, Alex X Lu, Naomi Caselli, et al. 2023. Asl citizen: a community-sourced dataset for advancing isolated sign language recognition. Advances in Neural Information Processing Syst… find
  4. Samuel Albanie, Gül Varol, Liliane Momeni, Hannah Bull, Triantafyllos Afouras, Himel Chowdhury, Neil Fox, Bencie Woll, Rob Cooper, Andrew Mc-Parland, et al. 2021. Bbc-oxford british sign language dataset. arXiv preprint arXiv:2111.03635. find
  5. Sarah Alyami, Hamzah Luqman, and Mohammad Hammoudeh. 2024. Isolated arabic sign language recognition using a transformer-based model and landmark keypoints. ACM Transactions on Asian and Low-Resource Language Information Processing, 23(1):1–19. find
  6. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learni…
  7. Joao Carreira and Andrew Zisserman. 2017. Quo vadis, action recognition? a new model and the kinetics dataset. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6299–6308. find
  8. Jérôme Fink, Benoît Frénay, Laurence Meurant, and Anthony Cleve. 2021. Lsfb-cont and lsfb-isol: Two new datasets for vision-based sign language recognition. In 2021 International Joint Conference on Neural Networks (IJCNN). find
  9. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework Jerome Fink, Pierre Poitier, Maxime André, Loup Meurice, Benoît Frénay, Anthony Cleve, Bruno Dumas, and Laurence Meurant. 2023. Sign language-to-text dictionary with lightweight transformer models. In IJCAI,… find
  10. Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. i6doc. com. find
  11. Ariel Basso Madjoukeng, Jérôme Fink, Pierre Poitier, Edith Belise Kenmogne, and Benoit Frenay. 2026. SSL-SLR: Self-supervised representation learning for sign language recognition. Transactions on Machine Learning Research. find
  12. Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. 2020. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neural information proces… find
  13. Ariel Basso Madjoukeng, Edith Bélise Kenmogne, Pierre Poitier, Benoît Frénay, and Jérôme Fink. 2025b. Local-global data augmentation for contrastive learning in static sign language recognition. In International Symposium on Intelligent Data Analysis, pages 54–66. Springer. find
  14. Shester Gueuwou, Xiaodan Du, Greg Shakhnarovich, and Karen Livescu. 2025. Signmusketeers: An efficient multi-stream approach for sign language translation at scale. In Findings of the Association for Computational Linguistics: ACL 2025, pages 22506–22521. find
  15. Philip Andrew Mansfield, Arash Afkanpour, Warren Richard Morningstar, and Karan Singhal. 2023. Random field augmentations for self-supervised representation learning. CoRR. find
  16. Sarfaraz Masood, Adhyan Srivastava, Harish Chandra Thuwal, and Musheer Ahmad. 2018. Real-time sign language gesture (word) recognition from video sequences using cnn and rnn. In Intelligent Engineering Informatics: Proceedings of the 6th International Conference on FICTA, pages 623–632. Springer. find
  17. Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729–9738.
  18. Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional lstm-crf models for sequence tagging. arXiv preprint arXiv:1508.01991.
  19. Katerina Papadimitriou, Galini Sapountzaki, Kyriaki Vasilaki, Eleni Efthimiou, Stavroula-Evita Fotinea, and Gerasimos Potamianos. 2024. A large corpus for the recognition of greek sign language gestures. Computer Vision and Image Understanding, 249:104212. find
  20. Zifan Jiang, Gerard Sant, Amit Moryossef, Mathias Müller, Rico Sennrich, and Sarah Ebling. 2024. Signclip: Connecting text and sign language by contrastive learning. arXiv preprint arXiv:2407.01264. find
  21. Pierre Poitier, Jérôme Fink, and Benoît Frénay. 2024. Towards better transition modeling in recurrent neural networks: The case of sign language tokenization. Neurocomputing, 567:127018. find
  22. Gregory Koch, Richard Zemel, Ruslan Salakhutdinov, et al. 2015. Siamese neural networks for one-shot image recognition. In ICML deep learning workshop, volume 2, pages 1–30. Lille. find
  23. Katrin Renz, Nicolaj C Stache, Samuel Albanie, and Gül Varol. 2021. Sign language segmentation with temporal convolutional networks. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2135–2139. IEEE. find
  24. Nico Lingg, Miguel Sarabia, Luca Zappella, and Barry-John Theobald. 2022. Contrastive self-supervised learning for skeleton representations. arXiv preprint arXiv:2211.05304. — [arXiv:2211.05304](https://arxiv.org/abs/2211.05304) find
  25. Ryan Wong, Necati Cihan Camgoz, and Richard Bowden. 2025. Signrep: Enhancing self-supervised sign representations. arXiv preprint arXiv:2503.08529. — [arXiv:2503.08529](https://arxiv.org/abs/2503.08529) find
  26. Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris McClanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo-Ling Chang, Ming Guang Yong, Juhyun Lee, et al. 2019. Mediapipe: A framework for building perception pipelines. arXiv preprint arXiv:1906.08172. — [arXiv:1906.08172](https://arxiv.org/abs/1906.08172) find
  27. Ariel Basso Madjoukeng, Jerome Fink, Pierre Poitier, Edith Belise Kenmogne, and Benoît Frénay. 2025a. Benchmarking data augmentation for contrastive learning in static sign language recognition. In ESANN 2024: 32nd European find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/madjoukeng-2026-leveraging-unannotated-sign-language-data/