Elamur Research⌕
☰
home / papers / 2026 / COACH Meets QUORUM: A Framework and Pipeline for Aligning User, Expert, and Developer Perspectives in LLM-Generated Health Counselling

COACH Meets QUORUM: A Framework and Pipeline for Aligning User, Expert, and Developer Perspectives in LLM-Generated Health Counselling

Paper · 2026 · Yee Man Ng, Bram van Dijk, Pieter Beynen et al.

Summary

Apps that log sleep, mood and daily activities could give cancer patients and survivors personalised lifestyle advice. Building such systems is hard, and evaluating them is harder, because existing evaluations tend to be fragmented and serve only one group of stakeholders. The authors make two contributions.

  • QUORUM is an evaluation framework that combines three views:
    • developers, who measure technical reliability (faithfulness, completeness, hallucination);
    • users, who judge relevance, tone, length and whether they would act on the advice;
    • medical experts, who judge correctness, tone and length.
  • COACH is an LLM pipeline built on Retrieval Augmented Generation. It links patterns found in a user's diary data with passages retrieved from the validated Dutch cancer information source kanker.nl. It was built for the Healthy Chronos diary app.

When QUORUM was applied to COACH, all three groups broadly agreed that the advice was relevant, of good quality and reliable. They disagreed on tone and on how much pattern-extraction errors and ungrounded statements matter. The work argues that health language technology meant for real-world use needs evaluation by several kinds of stakeholders.

Key points

  • Evaluators: COACH was assessed by 19 Dutch Healthy Chronos users, 6 information specialists from kanker.nl and 2 developers. QUORUM produces a per-perspective grand average, intended as a quick check during iterative development.
  • Faithfulness and completeness: 79% of claims about user data matched that user's records. For 97% of claims, the relevant information had been retrieved.
  • Hallucination: 22% of contextualisation statements could not be traced to the kanker.nl knowledge base. The authors count these as hallucinations, even though none were judged harmful.
  • Divergence: Users and experts differed on the preferred tone of the advice. Developers were more sensitive than the other groups to pattern-extraction errors and hallucinations.
  • Use case: Healthy Chronos is a Dutch diary app that tracks sleep, activities, mood and user-defined goals. A typical query is how to improve one's sleep.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Leiden UniversityLeiden · Netherlands32 works · 1,074 citations🤝Leiden University Medical CenterLeiden · Netherlands4 works · 2 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Medicine · 241 worksHealth Informatics · 184 worksHealth Professions · 69 worksHealth Information Management · 50 worksGeneral Health Professions · 14 works

🧭 Topics 1Research subjects and languages the work is about.Language models & pretraining · 3,320 works

Cite

Cite as Yee Man Ng, Bram van Dijk, Pieter Beynen, Otto Boekesteijn, Joris Jansen, Gerard van Oortmerssen, Max J. van Duijn and Marco Spruit. 2026. COACH Meets QUORUM: A Framework and Pipeline for Aligning User, Expert, and Developer Perspectives in LLM-Generated Health Counselling. Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.63317/4x7koa8w32ny

@inproceedings{ng2026coach,
  title = {{COACH Meets QUORUM: A Framework and Pipeline for Aligning User, Expert, and Developer Perspectives in LLM-Generated Health Counselling}},
  author = {Yee Man Ng, Bram {van Dijk}, Pieter Beynen, Otto Boekesteijn, Joris Jansen, Gerard {van Oortmerssen}, Max J. {van Duijn} and Marco Spruit},
  booktitle = {Proceedings of the Third Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC 2026},
  year = {2026},
  doi = {10.63317/4x7koa8w32ny},
  url = {https://doi.org/10.63317/4x7koa8w32ny}
}

References

3 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Tatsuo Akechi, Akira Kugaya, Hitoshi Okamura, Shigeto Yamawaki, and Yosuke Uchitomi. 1999. Fatigue and its associated factors in ambulatory cancer patients: a preliminary study. Journal of pain and symptom management, 17(1):42–48. \#\# Acknowledgments Alexander Bondarenko, Ekaterina Shirshakova, Marin… find
  2. We thank the users of Healthy Chronos who contributed as participants to this study. We are grateful to the cancer information experts from kanker.nl for their time and effort in evaluating the responses. We also thank Healthy Chronos for the collaboration and access to the valuable platform. 8. find
  3. Nattapat Boonprakong. 2025. Quantifying and Understanding the Effects of Cognitive Biases in Human-Computer Interaction. Ph.D. thesis, The University of Melbourne, Melbourne, Australia. \#\# Limitations find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/ng-2026-coach-meets-quorum-framework-pipeline/