Elamur Research⌕
☰
home / papers / 2026 / SASTA Self Assessment: An efficient human-in-the-loop strategy for developmental and pathological language analysis

SASTA Self Assessment: An efficient human-in-the-loop strategy for developmental and pathological language analysis

Paper · 2026 · Jan Odijk, Jelte van Boheemen, Xander Vertegaal et al.

Summary

SASTA is a Dutch tool that semi-automatically analyses transcripts of spontaneous speech. It supports clinical methods for assessing language in young children and in people with aphasia. It corrects deviant child or patient language and parses the candidate variants with Alpino. It then picks the best parse using roughly 30 evaluation criteria and maps the parse back onto the original words. Language-measure codes are computed as queries over that parse. Average F1 lies between 88.5% and 94.4%, but some samples score much lower. Experts therefore still need to check results, and reviewing whole samples cancels much of the time saved.

The authors add a self-assessment step. SASTA flags the utterances whose analysis is likely wrong, so a human revises only those. The flagging uses two classes of criteria. The authors also set a target F1 for samples after self-assessment and expert revision, and found this target is in principle attainable for all datasets and methods. On training and test data, the approach works promisingly for ASTA and STAP, while TARSP still needs substantial improvement. This matters because it could make clinical language sample analysis much faster.

Key points

  • Supported methods:
    • TARSP, for children aged 1–4.
    • STAP, for children aged 4–6.
    • ASTA, for patients with aphasia.
    • Each sample has 35–50 utterances, and codes count lexical, morphological, syntactic and error phenomena.
  • Pipeline: deviancy detection, then alternative generation (including a child-language spelling corrector), then Alpino parsing of all variants. About 30 parse evaluation criteria choose the best parse, for example penalising dp (discourse part) relations. A "backplacement" step restores the original words.
  • Baseline performance: average F1 is 88.5%–94.4%, and individual samples score as low as 77.6%. This is not reliable enough for fully automatic use.
  • Self-assessment results: promising for ASTA and STAP datasets. TARSP still falls short, and the authors outline directions for improving it.
  • Availability: SASTA is publicly available at sasta.hum.uu.nl, and its source code is on GitHub under the BSD 3-Clause License.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Utrecht UniversityUtrecht · Netherlands39 works · 1,603 citations

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksDevelopmental and Educational Psychology · 499 works

Cite

Cite as Jan Odijk, Jelte van Boheemen, Xander Vertegaal, Tessel Boerma and Marijn Schraagen. 2026. SASTA Self Assessment: An efficient human-in-the-loop strategy for developmental and pathological language analysis. Proceedings of the 8th Workshop on Clinical Natural Language Processing (Clinical NLP) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/22ko26n2nfwv

@inproceedings{odijk2026sasta,
  title = {{SASTA Self Assessment: An efficient human-in-the-loop strategy for developmental and pathological language analysis}},
  author = {Jan Odijk, Jelte {van Boheemen}, Xander Vertegaal, Tessel Boerma and Marijn Schraagen},
  booktitle = {Proceedings of the 8th Workshop on Clinical Natural Language Processing (Clinical NLP) @ LREC 2026},
  year = {2026},
  doi = {10.63317/22ko26n2nfwv},
  url = {https://doi.org/10.63317/22ko26n2nfwv}
}

References

29 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Bhasha Agrawal, Rahul Agarwal, Samar Husain, and Dipti M. Sharma. 2013. An automatic approach to treebank error detection using a dependency parser. In Alexander Gelbukh, editor, Computational Linguistics and Intelligent Text Processing. find
  2. CICLing 2013. Lecture Notes in Computer Science, vol. 7816, pages 294–303. Springer, Berlin, Heidelberg. find
  3. Bharat Ram Ambati, Mridul Gupta, Samar Husain, and Dipti Misra Sharma. 2010. A high recall error identification tool for Hindi treebank validation. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC’10), Valletta, Malta. European Language Resources Asso… find
  4. Gosse Bouma, Gertjan van Noord, and Robert Malouf. 2001. Alpino: Wide-coverage computational analysis of Dutch. Language and Computers, 37(1):45–59. find
  5. Markus Dickinson and Amber Smith. 2011. Detecting dependency parse errors with minimal resources. In Proceedings of the 12th International Conference on Parsing Technologies, pages 241–252, Dublin, Ireland. Association for Computational Linguistics. find
  6. Inge Klatte, Vera Van Heugten, Rob Zwitserlood, and Ellen Gerrits. 2022. Language sample analysis in clinical practice: Speech-language pathologists’ barriers, facilitators, and needs. Language, Speech, and Hearing Services in Schools, 53:1– 16. find
  7. Katarzyna Krasnowska and Adam Przepiórkowski. 2013. Detecting syntactic errors in dependency treebanks for morphosyntactically rich languages. In Language Processing and Intelligent Information Systems – 20th International Conference, IIS 2013 (selected papers), pages 69–79. Springer. find
  8. Brian MacWhinney. 2000. The CHILDES Project: Tools for Analyzing Talk, 3 edition. Lawrence Erlbaum Associates, Mahwah, NJ. find
  9. Elsbeth Boxum, Fennetta van der Scheer, and Mariëlle Zwaga. 2013. Analyse voor Spontane Taal bij Afasie. Standaard in samenwerking met de VKL. VKL. https://klinischelinguistiek.nl/ uploads/201307asta4eversie.pdf. ↗
  10. Jan Odijk. 2020. Towards semi-automatic analysis of spontaneous language for Dutch. In Selected papers from the CLARIN Annual Conference 2020, volume 180 of Linköping Electronic Conference Proceedings, pages 165–175. Linköping University Press, Linköping, Sweden. find
  11. Adriane Boyd, Markus Dickinson, and W. Detmar Meurers. 2008. On detecting errors in dependency treebanks. Research on Language and Computation, 6(2):113–137. find
  12. Jan Odijk, Alexis Dimitriadis, Martijn Van der Klis, Marjo Van Koppen, Meie Otten, and Remco Van der Veen. 2018. The AnnCor CHILDES Treebank. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Associa… find
  13. Daniël de Kok, Jianqiang Ma, and Gertjan van Noord. 2009. A generalized method for iterative error mining in parsing results. In Proceedings of the 2009 Workshop on Grammar Engineering Across Frameworks (GEAF 2009), pages 71– 79, Suntec, Singapore. Association for Computational Linguistics.
  14. Markus Dickinson. 2011. Detecting ad hoc rules for treebank development. Linguistic Issues in Language Technology (LiLT), 4(3). find
  15. Markus Dickinson and W. Detmar Meurers. 2003. Detecting inconsistencies in treebanks. In Proceedings of the Second Workshop on Treebanks and Linguistic Theories (TLT 2003), pages 45– 56, Växjö, Sweden. Växjö University Press. find
  16. Markus Dickinson and W. Detmar Meurers. 2005. Detecting errors in discontinuous structural annotation. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05), pages 322–329, Ann Arbor, Michigan. Association for Computational Linguistics. find
  17. Jan Odijk, Margo Zwitserlood-Nijenhuis, Rob Zwitserlood, Jelte van Boheemen, Anouk Scheffer, and Frank Wijnen. 2025. Semi-automatische spontane taalanalyse (SASTA): Op weg naar een efficiënte methode voor de bepaling van het taal-profiel van kinderen met TOS. Stem-, Spraak-en Taalpathologie, 30:79–1… find
  18. N. Oostdijk, W. Goedertier, F. Van Eynde, L. Boves, J.P. Martens, M. Moortgat, and H. Baayen. 2002. Experiences from the Spoken Dutch Corpus project. In M. González Rodriguez and C. Paz Suárez Araujo, editors, Proceedings of the third International Conference on Language Resources and Evaluation (LR… find
  19. Benoît Sagot and Éric de la Clergerie. 2006. Error mining in parsing results. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 329–336, Sydney, Australia. Association for Computational Lin… find
  20. Liesbeth Schlichting. 2005. TARSP: Taal Analyse Remediëring en Screening Procedure. Taalont-wikkelingsschaal van Nederlandse kinderen van 1-4 jaar, 7th edition. Pearson, Amsterdam. find
  21. Alexander Volokh and Günter Neumann. 2011. Automatic detection and correction of errors in dependency treebanks. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (short papers), pages 346–350, Portland, Oregon, USA. Association f…
  22. Liesbeth Schlichting. 2017. TARSP: Taalontwikke-lingsschaal van Nederlandse kinderen van 1-4 jaar met aanvullende structuren tot 6 jaar, 8th edition. Pearson, Amsterdam. find
  23. Kanta Suzuki, Yoshihide Kato, and Shigeki Matsubara. 2016. Correcting errors in a treebank based on tree mining. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 1540– 1545, Portorož, Slovenia. European Language Resources Association (ELRA). find
  24. Hans van Halteren. 2000. The detection of inconsistency in manually tagged text. In Proceedings of the COLING-2000 Workshop on Linguistically Interpreted Corpora, pages 48–55, Centre Universitaire, Luxembourg. International Committee on Computational Linguistics. find
  25. Margreet van Ierland, Jeannette Verbeek, and Leen van den Dungen. 2008. Spontane Taal Analyse Protocol. Handleiding van het STAP-instrument. UvA, Amsterdam. find
  26. Gertjan van Noord. 2004. Error mining for wide-coverage grammar engineering. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04), pages 446–453, Barcelona, Spain. find
  27. Gertjan van Noord. 2006. At last parsing is now operational. In Actes de la 13ème conférence sur le Traitement Automatique des Langues Naturelles. Conférences invitées, pages 20–42, Leuven, Belgique. ATALA. find
  28. Gertjan van Noord, Gosse Bouma, Frank Van Eynde, Daniël de Kok, Jelmer van der Linde, Ineke Schuurman, Erik Tjong Kim Sang, and Vincent Vandeghinste. 2013. Large scale syntactic annotation of written Dutch: Lassy. In Peter Spyns and Jan Odijk, editors, Essential Speech and Language Technology for Du… find
  29. Jeannette Verbeek, Leen van den Dungen, and Anne Baker. 2007. Spontane Taal Analyse Protocol. Verantwoording van het STAP-instrument, ontwikkeld door Margreet van Ierland. UvA. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/odijk-2026-sasta-self-assessment-efficient-human/