📄 CoTrust: Privacy-Preserving Collaboration Between Large and Small Language Models in Trusted Execution Environments

Summary

LLM services routinely receive prompts that contain personal or confidential data. A Trusted Execution Environment (TEE) could keep such data safe, but TEEs lack the memory and compute to host billion-parameter models. Masking sensitive spans before sending a prompt out protects privacy, but it removes information the model needs. CoTrust addresses this by splitting the work between two models:

• A small language model (SLM) runs inside the TEE and sees the full input.
• An external LLM, assumed honest-but-curious, only receives several differently de-identified versions of the prompt.

The LLM drafts one candidate answer per version. The TEE then aligns the placeholders in these drafts, and the LLM merges the aligned drafts into a single consensus scaffold. The scaffold holds the reasoning and answer structure but no private details. Finally, the SLM completes the answer by grounding the scaffold in the original input.

The authors tested CoTrust on QA (MedQA, SQuAD) and summarization (DialogSum, SAMSum). It nearly matches an unrestricted LLM, outperforms earlier privacy-preserving baselines, keeps leakage close to single-mask methods and runs efficiently on Intel TDX. This makes it practical to deploy strong assistants on sensitive data.

Key points

• Claimed novelty: the authors present this as the first privacy-preserving LLM–SLM collaboration in a TEE setting, implemented on an Intel TDX-based TEE.
• Privacy evaluation:
  • Qwen3-Max served as both privacy-content extractor and judge.
  • On QA, CoTrust kept PPS above 0.82 and ESR under 2%.
  • On summarization, PPS was about 0.90 and ESR at most 1.35%.
  • Compared with the Spacy and Presidio single-mask baselines, PPS dropped by only about 0.003–0.045.
• Ablation on SQuAD (Qwen3-4B/8B):
  • Full CoTrust reaches 54.68 EM / 63.75 F1.
  • Giving the LLM a single masked view drops this to 42.07 / 51.87.
  • Adding test-time scaling to the single view only recovers to 50.43 / 58.91.
• Other ablated variants:
  • Removing the scaffold gives 45.93 / 56.71.
  • Skipping canonicalization gives 49.84 / 58.76.
  • Passing raw multiple candidates to the SLM gives 53.62 / 61.67.
  • Running scaffold induction in the SLM gives 48.68 / 58.19.
• Efficiency: latency and memory overhead are modest compared with an SLM-only baseline and far below running a full LLM inside the TEE.