Elamur Research⌕
☰
home / papers / 2026 / Decomposing Sign Language Movements: A Multi-Band Visualization Method for Articulatory Analysis

Decomposing Sign Language Movements: A Multi-Band Visualization Method for Articulatory Analysis

Summary

Sign articulation is layered: several gestures, such as arm displacement, wrist rotation and finger changes, can happen at once. Raw video makes this structure hard to study. The authors propose a method that works on ordinary 2D video. It uses the MediaPipe Hand Landmarker to locate hand keypoints and then separates motion along the articulatory chain. Wrist rotation is factored out of arm movement, and finger motion is factored out of overall hand movement. For each component, the method computes velocity and direction. These channels are stacked as parallel, time-aligned tracks in a plot the authors call a "bandogram." An open-source web viewer links the bandogram to the video, so clicking a stretch of the plot shows the matching signing. The method is demonstrated qualitatively on isolated signs and on continuous signing. It exposes bimanual coordination, internal movements, and the boundary between linguistic material and non-linguistic preparation or relaxation gestures. The paper reports no quantitative evaluation. Its value lies in giving phonology and prosody researchers a low-cost way to examine rhythm, syllable structure and timing in existing sign language video, without motion-capture hardware.

Key points

  • Tracking input: MediaPipe Hand Landmarker yields 21 three-dimensional landmarks per hand, covering the wrist, all finger joints and the MCP joints. From these, the authors derive a compact set of position and orientation features.
  • Decomposition: Motion is split into three components: the hand's path through space, wrist rotation, and individual finger movement. Velocity and direction are computed separately for each articulator.
  • Visualization and tool: The bandogram is paired with an interactive viewer synchronized to the video. The code is freely available at github.com/agarsev/bandogram.
  • Examples: The demonstrations include a Spanish Sign Language (LSE) sign, Cabeza Cuadrada ("Block-head"), which combines external displacements with a handshape change and wrist rotation. Signing data came from informants at "Idiomas Complutense."
  • Positioning: The approach differs from holistic PCA decomposition of motion-capture data (Bigand et al., 2021) and from generative modelling of hand trajectories (Chassat et al., 2022). It aims for a linguistically motivated split of the articulatory chain that works on plain 2D video.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

Fields: Linguistics & languages

Cite

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/32sdurbs4fio

@inproceedings{sevilla2026decomposing,
  title = {{Decomposing Sign Language Movements: A Multi-Band Visualization Method for Articulatory Analysis}},
  author = {Sevilla, Antonio F. G. and Lahoz-Bengoechea, José María},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/32sdurbs4fio},
  url = {https://doi.org/10.63317/32sdurbs4fio}
}

References

28 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Sunusi Bala Abdullahi and Kosin Chamnongthai. 2022. American Sign Language Words Recognition Using Spatio-Temporal Prosodic and Angle Features: A Sequential Learning Approach. IEEE Access, 10:15911–15923. Conference Name: IEEE Access. find
  2. Mark Aronoff, Irit Meir, and Wendy Sandler. 2005. find
  3. The Paradox of Sign Language Morphology. Language, 81(2):301–344. find
  4. Félix Bigand, Elise Prigent, Bastien Berret, and Annelies Braffort. 2021. Decomposing spontaneous sign language into elementary movements: A principal component analysis-based approach. PLOS ONE, 16(10). find
  5. Ángel Luis Herrero Blanco. 2009. Gramática didáctica de la lengua de signos española (LSE). Sm. find
  6. Joshua D. Borneman, Evie Malaia, and Ronnie B. Wilbur. 2018. Motion characterization using optical flow and fractal complexity. Journal of Electronic Imaging, 27(5). find
  7. Diane Brentari, Jordan Fenlon, and Kearsy Cormier. 2018. Sign Language Phonology. In Oxford Research Encyclopedia of Linguistics. Oxford University Press. find
  8. Carl Börstell. 2023. Extracting Sign Language Articulation from Videos with MediaPipe. In Proceedings of the 24th Nordic Conference on Computational Linguistics (NoDaLiDa), pages 169–178, Tórshavn, Faroe Islands. University of Tartu Library. find
  9. Perrine Chassat, Juhyun Park, and Nicolas Brunel. 2022. Analysis of variability in sign language hand trajectories: Development of generative model. In Proceedings of the 8th International Conference on Movement and Computing, pages 1–8, Chicago IL USA. ACM. find
  10. Jordan Fenlon and Diane Brentari. 2021. Prosody: Theoretical and experimental perspectives. In Josep Quer, Roland Pfau, and Annika Herrmann, editors, The Routledge Handbook of Theoretical and Experimental Sign Language Research. find
  11. Thomas Hanke, Marc Schulder, Reiner Konrad, and Elena Jahn. 2020. Extending the public dgs corpus in size and depth. In Proceedings of the LREC2020 9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Cha… find
  12. Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. 2024. CoTracker: It is better to track together. In Computer Vision – ECCV 2024. find
  13. Chemuttaai Koech. 2007. A kinematic analysis of sign language. Master’s thesis, New Jersey Institute of Technology. find
  14. Scott K. Liddell and Robert E. Johnson. 1989. American Sign Language: The Phonological Base. Sign Language Studies, 64(1):195–277. find
  15. Camillo Lugaresi, Jiuqiang Tang, Hadon Nash, Chris McClanahan, Esha Uboweja, Michael Hays, Fan Zhang, Chuo-Ling Chang, Ming Yong, Juhyun Lee, et al. 2019. Mediapipe: A framework for perceiving and processing reality. In Third workshop on computer vision for AR/VR at IEEE computer vision and pattern… find
  16. Irma Muñoz Baell. 1999. ¿Cómo se articula la LSE? CNSE, Spain. find
  17. Takehiko Ohkawa, Ryosuke Furuta, and Yoichi Sato. 2023. Efficient Annotation and Learning for 3D Hand Pose Estimation: A Survey. International Journal of Computer Vision. find
  18. Jefferson Rodríguez, Juan Chacón, Edgar Rangel, Luis Guayacán, Claudia Hernández, Luisa Hernández, and Fabio Martínez. 2020. Understanding motion in sign language: A new structured translation dataset. find
  19. Abraham Savitzky and Marcel JE Golay. 1964. Smoothing and differentiation of data by simplified least squares procedures. Analytical chemistry, 36(8):1627–1639. find
  20. Antonio F. G. Sevilla, José María Lahoz-Bengoechea, and Alberto Diaz. 2024. Automated Extraction of Prosodic Structure from Unannotated Sign Language Video. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), p… find
  21. Kelly M Stack. 1988. Tiers and Syllable Structure in American Sign Language: Evidence from Phonotactics. Master’s thesis, University of California, Los Angeles. find
  22. William C. Stokoe. 1960. Sign Language Structure: An Outline of the Visual Communication Systems of the American Deaf, volume 8 of Studies in Linguistics: Occasional Papers. University of Buffalo, Buffalo, NY. find
  23. Martha E Tyrone, Hosung Nam, Elliot Saltzman, Gaurav Mathur, and Louis Goldstein. 2010. Prosody and Movement in American Sign Language: A Task-Dynamics Approach. In Proceedings of Speech Prosody 2010, pages 1–4, Chicago, IL. ISCA. find
  24. Andrey Vakunov, Chuo-Ling Chang, Fan Zhang, George Sung, Matthias Grundmann, and Valentin Bazarevsky. 2020. Mediapipe hands: On-device real-time hand tracking. In Workshop on Computer Vision for Augmented and Virtual Reality at CVPR 2020. find
  25. Ronnie B Wilbur and George D Allen. 1991. Perceptual evidence against internal structure in American Sign Language syllables. Language and Speech, 34(1):27–46. find
  26. Ronnie B. Wilbur and Aleix M. Martinez. 2002. Physical correlates of prosodic structure in American Sign Language. In Chicago Linguistic Society, volume 38, pages 693–704. find
  27. Ronnie B Wilbur and Howard N Zelaznik. 1997. Kinematic correlates of stress and position in asl. In Annual Meeting of the Linguistic Society of America (LSA), Chicago, Illinois. find
  28. Kayo Yin, Amit Moryossef, Julie Hochgesang, Yoav Goldberg, and Malihe Alikhani. 2021. Including Signed Languages in Natural Language Processing. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language… find

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/sevilla-2026-decomposing-sign-language-movements-multi/