📄 Diagnosable ColBERT: Debugging Late-Interaction Retrieval Models Using a Learned Latent Space as Reference

Summary

ColBERT-style late-interaction retrievers are often called interpretable because they show which query token matched which document token. The author argues that this is not enough for clinical search. These scores justify a successful match after the fact, but they say little about why a relevant document was missed. They also cannot show whether the model reliably grasps a concept and its qualifiers (negation, temporality, uncertainty, experiencer) across different wordings and contexts. This short position paper proposes Diagnosable ColBERT. Token embeddings are mapped through pre-projection adapters into a reference latent space shaped by clinical knowledge and expert similarity constraints, such as BioLORD-like spaces. Retrieval vectors are then derived as a lower-dimensional projection of this inspectable representation. Testers can see where query and document tokens fall in the space and tell concept-grounding errors apart from contextual misreadings, sometimes without issuing any query. The paper reports no quantitative evaluation. Its contribution is the diagnostic framing plus practitioner experience. This matters because it turns retrieval misses into targeted data-curation decisions.

Key points

• Published at the BioNLP 2026 workshop (pages 713–717) by François Remy (Parallia Healthcare). It is a position paper with no benchmark numbers.
• The architecture couples diagnosis and retrieval: the diagnosed embedding lives in the reference space, and the ranking embedding is a down-projection of it.
• Worked example: the query bartonellosis misses a report that mentions only CSD (cat scratch disease). Checking each side's placement in the latent space shows whether the fault lies in query grounding, document-side abbreviation handling, or both.
• Query-free diagnosis is also possible. In a dialogue where a patient reports an allergy to ranitidine, the tester can check whether the token encodes the allergy context or only the drug identity.
• The author reports that debugging interfaces built on this idea guided the move from ClinicalEncoder25 to ClinicalEncoder26AM. Public demos are at demo26.parallia.eu and text2json.parallia.eu.