Elamur Research⌕
☰
home / papers / 2026 / Comparing Computer Vision Instruments for Eye Blink Analysis

Comparing Computer Vision Instruments for Eye Blink Analysis

Paper · 2026 · Margaux Susman, Carla Miquel Blasco and Jan Bulla

Summary

Computer vision tools are increasingly used to measure non-manual signals in sign languages, but it is unclear whether different tools produce comparable kinematic measurements. The authors tested this for eye blinks by running MediaPipe, OpenFace, InsightFace and 3DDFA on the same French Sign Language recordings from the Dicta-Sign-LSF-v2 corpus. They manually annotated blinks for their function and computed eyelid openness with the Eye Aspect Ratio (EAR). From smoothed EAR curves they derived blink amplitude and average velocity. Linear mixed-effects models then compared blink categories while adjusting for head pose (Pitch, Roll, Yaw).

The tools disagreed substantially:

  • InsightFace and MediaPipe tended to show negative category effects, whereas 3DDFA sometimes showed effects in the opposite direction.
  • The importance and direction of the head-pose predictors also varied by tool.

Because the choice of tool can change the statistical conclusions, the authors warn that apparent linguistic patterns may partly reflect artefacts of the measurement pipeline. This matters for anyone scaling up corpus-based analysis of non-manual markers.

Key points

  • Data: 8 Dicta-Sign-LSF-v2 videos (dialogues about European travel) were annotated in ELAN 6.2. Of 1078 annotated blinks, the 650 with a single function were analysed.
  • Blink categories:
    • Linguistic: backchanneling, prosodic/turn-taking, and sign-aligned/list-marking.
    • Non-linguistic: eye-watering and gaze-direction-change.
  • Pipeline:
    • EAR was computed from six eyelid landmarks per eye (Soukupova and Cech, 2016).
    • Quintic smoothing splines were fitted with ss from the R package npreg, with λ chosen by GCV.
    • Models were fitted with lme() from nlme, using a random intercept per signer.
    • Predictors were added stepwise and compared with likelihood ratio tests or AIC.
  • Scale differences: Velocity estimates differed widely between tools. For backchanneling blinks, the estimated mean velocity was about 0.02 for InsightFace, 0.09 for OpenFace, 0.58 for MediaPipe, and negative (−2.23) for 3DDFA.
  • Head pose: Pitch was strongly positive with MediaPipe but negative with InsightFace and in some 3DDFA models. The work was funded by the ERC project NONMANUAL (101,039,378).

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of BergenBergen · Norway9 works · 583 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksExperimental and Cognitive Psychology · 870 worksHuman-Computer Interaction · 141 works

Cite

Cite as Margaux Susman, Carla Miquel Blasco and Jan Bulla. 2026. Comparing Computer Vision Instruments for Eye Blink Analysis. Proceedings of the LREC 2026 12th Workshop on the Representation and Processing of Sign Languages: Language in Motion.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/2vzsgmq46vrk

@inproceedings{susman2026comparing,
  title = {{Comparing Computer Vision Instruments for Eye Blink Analysis}},
  author = {Margaux Susman, Carla Miquel Blasco and Jan Bulla},
  booktitle = {Proceedings of the LREC 2026 12th Workshop on the Representation and Processing of Sign Languages: Language in Motion},
  year = {2026},
  doi = {10.63317/2vzsgmq46vrk},
  url = {https://doi.org/10.63317/2vzsgmq46vrk}
}

References

8 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Brandon Amos, Bartosz Ludwiczuk, Mahadev Satyanarayanan, et al. 2016. Openface: A general-purpose face recognition library with mobile applications. CMU School of Computer Science, 6(2):20. find
  2. Vincent Arel-Bundock, Noah Greifer, and Andrew Heiss. 2024. How to interpret statistical models using marginaleffects for R and Python. Journal of Statistical Software, 111(9):1–32. find
  3. Valentin Belissen, Annelies Braffort, and Michèle Gouiffès. 2020. Dicta-Sign-LSF-v2: Remake of a continuous French Sign Language dialogue corpus and a first baseline for automatic sign language processing. In 12th International Conference on Language Resources and Evaluation (LREC 2020), pages 6040–… find
  4. Annelies Braffort and Emilie Chételat-Pelé. 2012. Analysis and description of blinking in french sign language for automatic generation. In Gesture and Sign Language in Human-Computer Interaction and Embodied Communication: 9th International Gesture Workshop, GW 2011, Athens, Greece, May 25-27, 2011… find
  5. Jan Bulla and Vadim Kimmelman. 2026. Processing kinematics of nonmanual markers in r. find
  6. John E Delgado Gómez, Elena Muñoz España, Carlos F Rengifo Rodas, and Diego E Guzmán Villamarín. 2024. Proposal for a eye blink detection using mediapipe, eye aspect ratio and peak identification. In International Conference on NeuroRehabilitation, pages 345–349. Springer. find
  7. Jia Guo, Jiankang Deng, Alexandros Lattas, and Stefanos Zafeiriou. 2021. Sample and compu-Fixed effects (Intercept) Pitch Roll blink cat. (PROS-TURN) blink cat. (SIGN-LIC) find
  8. InsightFace (cat + P) OpenFace (cat) MediaPipe (P + cat + R) 3DDFA (cat + P + R) 0.021, p \< 0.001 −0.000, p = 0.065 0.089, p \< 0.001 −0.002, p = 0.050 −0.003, p = 0.088 0.282, p = 0.012 7.154, p \< 0.001 22.166, p = 0.023 −0.090, p = 0.012 −2.067, p \< 0.001 −2707, p = 0.003 −687, p = 0.005 0.595, p \<… find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/susman-2026-comparing-computer-vision-instruments-eye/