Elamur Research⌕
☰
home / papers / 2026 / J-Shuwa: A Large-Scale Web-Collected Japanese Sign Language-Japanese Parallel Corpus

J-Shuwa: A Large-Scale Web-Collected Japanese Sign Language-Japanese Parallel Corpus

Paper · 2026 · Junwen Mo, MinhDuc Vo, Noriki Nishida et al.

Summary

Japanese Sign Language (JSL) has seen little AI research, mainly because no large public parallel corpora exist. Expert-annotated corpora are high quality but too small for training. YouTube-SL-25 offers only 62 hours of JSL. The authors built J-Shuwa from YouTube JSL videos. They used both closed captions and hard-coded subtitles (HardSubs), a source earlier work mostly ignored because extracting burned-in text is hard. Japanese adds further difficulties: varied fonts and placements, vertical right-to-left text, and Furigana.

The pipeline works in four steps:

  • Collect videos from JSL-related channels and apply basic quality filters.
  • Keep signing content using an action recognition model.
  • Find subtitle regions with OCR, then segment and read them with VideoSubFinder plus OCR.
  • Clean the output with heuristic filters and targeted manual checks.

The final corpus has about 197K sentence pairs, roughly 300 hours, making it the largest public JSL dataset. Sign language translation models trained on it were tested on the JSL Dialogue Corpus, both zero-shot and after fine-tuning. The corpus proved useful, though faithful translation remains difficult.

Key points

  • J-Shuwa has ~197K JSL–Japanese pairs, ~300 hours, and a 31K vocabulary. The JSL portion of YouTube-SL-25 has 62 hours, ~40K sentences, and a ~13K vocabulary.
  • Sources: about 464 channels and ~72K raw videos, including content by deaf creators, instructional videos, and local government broadcasts. Sign-language songs were excluded.
  • Video filtering used SlowFast pretrained on Kinetics-400. A video was kept if at least 60% of its sampled clips were labeled "Sign Language Interpreting".
  • Only YouTube video IDs, segment timestamps, and extraction code are released (github.com/SpaJune/J-Shuwa). No videos or subtitles are redistributed.
  • Limitations: annotators were not Deaf experts, so JSL and Signed Japanese (MCJ) are mixed in the data. Some samples are noisy or misaligned.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓The University of TokyoTokyo · Japan90 works · 9,156 citations🔬National Institute of InformaticsTokyo · Japan37 works · 3,247 citations🔬RIKENWako · Japan26 works · 526 citations

 Fields, topics and concepts

📚 Shelves 7The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksDevelopmental and Educational Psychology · 499 worksHuman-Computer Interaction · 141 works

Cite

Cite as Junwen Mo, MinhDuc Vo, Noriki Nishida, Shin'ichi Satoh and Hideki Nakayama. 2026. J-Shuwa: A Large-Scale Web-Collected Japanese Sign Language-Japanese Parallel Corpus. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.findings-acl.1821

@inproceedings{mo2026j,
  title = {{J-Shuwa: A Large-Scale Web-Collected Japanese Sign Language-Japanese Parallel Corpus}},
  author = {Junwen Mo, MinhDuc Vo, Noriki Nishida, Shin'ichi Satoh and Hideki Nakayama},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1821},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1821}
}

References

45 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. MMAction2 Contributors. 2020. Openmmlab’s next generation video understanding toolbox and benchmark. https://github.com/open-mmlab/ mmaction2. ↗
  2. Samuel Albanie, Gül Varol, Liliane Momeni, Hannah Bull, Triantafyllos Afouras, Himel Chowdhury, Neil Fox, Bencie Woll, Rob Cooper, Andrew McParland, and Andrew Zisserman. 2021. Bbc-oxford british sign language dataset. CoRR, abs/2111.03635. find
  3. Mathieu De Coster, Dimitar Shterionov, Mieke Van Herreweghe, and Joni Dambre. 2024. Machine translation from signed to spoken languages: state of the art and challenges. Univers. Access Inf. Soc., 23(3):1305–1331. find
  4. Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, and 45 others. 2025. Qwen3-vl technical report. Preprint, ar…
  5. Mayumi Bono, Kouhei Kikuchi, Paul Cibulka, and Yutaka Osugi. 2014. A colloquial corpus of Japanese Sign Language: Linguistic resources for observing sign language conversations. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 1898–1904, Reyk… find
  6. Mayumi Bono and Yutaka Osugi. 2026. A Colloquial Corpus of Sign Languages in Japan (JSL). Informatics Research Data Repository, National Institute of Informatics (dataset). find
  7. Danielle Bragg, Oscar Koller, Mary Bellard, Larwan Berke, Patrick Boudreault, Annelies Braffort, Naomi Caselli, Matt Huenerfauth, Hernisa Kacorri, Tessa Verhoef, Christian Vogler, and Meredith Ringel Morris. 2019. Sign language recognition, generation, and translation: An interdisciplinary perspecti… find
  8. Cheng Cui, Ting Sun, Manhui Lin, Tingquan Gao, Yubo Zhang, Jiaxuan Liu, Xueqing Wang, Zelun Zhang, Changda Zhou, Hongen Liu, Yue Zhang, Wenyu Lv, Kui Huang, Yichao Zhang, Jing Zhang, Jun Zhang, Yi Liu, Dianhai Yu, and Yanjun Ma. 2025. Paddleocr 3.0 technical report. Preprint, arXiv:2507.05595. — [arXiv:2507.05595](https://arxiv.org/abs/2507.05595) find
  9. Amanda Cardoso Duarte, Shruti Palaskar, Lucas Ventura, Deepti Ghadiyaram, Kenneth DeHaan, Florian Metze, Jordi Torres, and Xavier Giró-i-Nieto. 2021. How2sign: A large-scale multimodal dataset for continuous american sign language. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR… find
  10. Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. 2019. Slowfast networks for video recognition. In 2019 IEEE/CVF International Conference on Computer Vision, ICCV 2019, Seoul, Korea (South), October 27 - November 2, 2019, pages 6201–6210. IEEE. find
  11. Albert Gu and Tri Dao. 2023. Mamba: Linear-time sequence modeling with selective state spaces. CoRR, abs/2312.00752. find
  12. Shester Gueuwou, Sophie Siake, Colin Leong, and Mathias Müller. 2023a. JWSign: A highly multilingual corpus of Bible translations for more diversity in sign language processing. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 9907–9927, Singapore. Association for Comp… find
  13. Shester Gueuwou, Kate Takyi, Mathias Müller, Marco Stanley Nyarko, Richard Adade, and Rose-Mary Owusuaa Mensah Gyening. 2023b. Afrisign: Machine translation for african sign languages. In Proceedings of the 4th Workshop on African Natural Language Processing, AfricaNLP@ICLR 2023, Kigali, Rwanda, May… find
  14. Matthew Honnibal, Ines Montani, Sofie Van Landeghem, and Adriane Boyd. 2020. spaCy: Industrial-strength Natural Language Processing in Python. find
  15. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net.
  16. Li Ji, Jiang Liu, and Shigeru Shimamoto. 2022. Recognition of japanese sign language by sensor-based data glove employing machine learning. In 4th IEEE Global Conference on Life Sciences and Technologies, LifeTech 2022, Osaka, Japan, March 7-9, 2022, pages 256–258. IEEE. find
  17. Abhinav Joshi, Susmit Agrawal, and Ashutosh Modi. 2023. ISLTranslate: Dataset for translating Indian Sign Language. In Findings of the Association for Computational Linguistics: ACL 2023, pages 10466–10475, Toronto, Canada. Association for Computational Linguistics. find
  18. Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, and 1 others. 2017. The kinetics human action video dataset. arXiv preprint arXiv:1705.06950. find
  19. Sang-Ki Ko, Chang Jo Kim, Hyedong Jung, and Choongsang Cho. 2019. Neural sign language translation based on human keypoint estimation. Applied sciences, 9(13):2683. find
  20. Oscar Koller, Jens Forster, and Hermann Ney. 2015. Continuous sign language recognition: Towards large vocabulary statistical recognition systems handling multiple signers. Comput. Vis. Image Underst., 141:108–125. find
  21. Dongxu Li, Cristian Rodriguez Opazo, Xin Yu, and Hongdong Li. 2020. Word-level deep sign language recognition from video: A new large-scale dataset and methods comparison. In IEEE Winter Conference on Applications of Computer Vision, WACV 2020, Snowmass Village, CO, USA, March 1-5, 2020, pages 1448–… find
  22. Kunchang Li, Yali Wang, Yizhuo Li, Yi Wang, Yinan He, Limin Wang, and Yu Qiao. 2023. Unmasked teacher: Towards training-efficient video foundation models. In IEEE/CVF International Conference on Computer Vision, ICCV 2023, Paris, France, October 1-6, 2023, pages 19891–19903. IEEE. find
  23. Zecheng Li, Wengang Zhou, Weichao Zhao, Kepeng Wu, Hezhen Hu, and Houqiang Li. 2025. Uni-sign: Toward unified sign language understanding at scale. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net. find
  24. Aobo Liang, Xingguo Jiang, Yan Sun, Xiaohou Shi, and Ke Li. 2024. Bi-mamba+: Bidirectional mamba for time series forecasting. arXiv preprint arXiv:2404.15772. — [arXiv:2404.15772](https://arxiv.org/abs/2404.15772) find
  25. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. find
  26. Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris McClanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo-Ling Chang, Ming Guang Yong, Juhyun Lee, Wan-Teh Chang, Wei Hua, Manfred Georg, and Matthias Grundmann. 2019. Mediapipe: A framework for building perception pipelines. CoRR, abs/1906.08172. — [arXiv:1906.08172](https://arxiv.org/abs/1906.08172) find
  27. Ryota Murai, Naoto Tsuta, Duk Shin, and Yousun Kang. 2025. Point-supervised japanese fingerspelling localization via hr-pro and contrastive learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). find
  28. Yuji Nagashima. 2021. Kogakuin university japanese sign language multi-dimensional database (kosign). https://doi.org/10.32130/rdata.5.1. Dataset. — [doi:10.32130/rdata.5.1](https://doi.org/10.32130/rdata.5.1) ↗
  29. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational…
  30. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186–191, Brussels, Belgium. Association for Computational Linguistics.
  31. Joseph Redmon, Santosh Kumar Divvala, Ross B. Girshick, and Ali Farhadi. 2016. You only look once: Unified, real-time object detection. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pages 779–788. IEEE Computer Society. find
  32. Ramon Sanabria, Ozan Caglayan, Shruti Palaskar, Desmond Elliott, Loïc Barrault, Lucia Specia, and Florian Metze. 2018. How2: A large-scale dataset for multimodal language understanding. CoRR, abs/1811.00347. find
  33. Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020. BLEURT: Learning robust metrics for text generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7881–7892, Online. Association for Computational Linguistics.
  34. Xin Shen, Shaozu Yuan, Hongwei Sheng, Heming Du, and Xin Yu. 2023. Auslan-daily: Australian sign language translation for daily communication and news. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans,… find
  35. Bowen Shi, Diane Brentari, Gregory Shakhnarovich, and Karen Livescu. 2022. Open-domain sign language translation learned from online video. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pag… find
  36. Jungpil Shin, Md Al, Abu Saleh, Kota Suzuki, and Koki Hirooka. 2024. Japanese sign language recognition by combining joint skeleton-based handcrafted and pixel-based deep learning features with machine learning classification. Computer Modeling in Engineering & Sciences, 139(3):2605. find
  37. Ariel E. Stassi, Yanina Boria, J. Matías Di Martino, and Gregory Randall. 2025. ilsu-t: an open dataset for uruguayan sign language translation. In 19th IEEE International Conference on Automatic Face and Gesture Recognition, FG 2025, Tampa/Clearwater, FL, USA, May 26-30, 2025, pages 1–10. IEEE. find
  38. Sihan Tan, Nabeela Khan, Zhaoyi An, Yoshitaka Ando, Rei Kawakami, and Kazuhiro Nakadai. 2024. A review of deep learning-based approaches to sign language processing. Adv. Robotics, 38(23):1649–1667. find
  39. Garrett Tanzer and Biao Zhang. 2025. Youtube-sl-25: A large-scale, open-domain multilingual sign language parallel corpus. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net. find
  40. Michael Tschannen, Alexey A. Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, Olivier J. Hénaff, Jeremiah Harmsen, Andreas Steiner, and Xiaohua Zhai. 2025. Siglip 2: Multilingual vision-language encoders with… find
  41. David Uthus, Garrett Tanzer, and Manfred Georg. 2023. Youtube-asl: A large-scale, open-domain american sign language-english parallel corpus. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA,… find
  42. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mT5: A massively multilingual pre-trained text-to-text transformer. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Ling…
  43. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 40 others. 2025. Qwen3 technical report. CoRR, abs/2505.09388.
  44. Aoxiong Yin, Zhou Zhao, Weike Jin, Meng Zhang, Xingshan Zeng, and Xiaofei He. 2022. MLSLT: towards multilingual sign language translation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pages 5099–5109. IEEE. find
  45. Hao Zhou, Wengang Zhou, Weizhen Qi, Junfu Pu, and Houqiang Li. 2021. Improving sign language translation with monolingual data by sign back-translation. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2021, virtual, June 19-25, 2021, pages 1316–1325. Computer Vision Foundation /… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/mo-2026-j-shuwa-large-scale-web/