Elamur Research⌕
☰
home / papers / 2026 / When Agents Look the Same: Quantifying Distillation-Induced Similarity in Tool-Use Behaviors

When Agents Look the Same: Quantifying Distillation-Induced Similarity in Tool-Use Behaviors

Paper · 2026 · Chenghao Yang, Yuning Zhang, Zhoufutu Wen et al.

Summary

Many LLM agents now behave almost identically, likely because they are distilled from a few strong teachers. Existing similarity measures compare static responses. They cannot separate steps a task forces on every model from optional choices that reveal a model's own habits. The authors propose two metrics to fill this gap. Response Pattern Similarity (RPS) has an LLM tag trajectory segments with five interaction stages (authentication, elicitation, execution, verification, notification). An LLM judge then rates style, structure and alignment, but only on stages both models share. Action Graph Similarity (AGS) represents tool calls as directed graphs. It scores agreement on optional tools (S node), ordering habits (S seq) and output reuse (S dep). The authors measure 18 models from 8 providers against Claude Sonnet 4.5 (thinking) on τ-Bench and τ2-Bench. Same-family models are clearly closer to each other. Kimi-K2 (thinking) resembles Claude more than some Anthropic models do. A controlled LoRA distillation run confirms that AGS tracks the actual teacher. The framework offers a practical way to detect hidden behavioral convergence, which weakens the independence of agents.

Key points

  • Same-family model pairs get AGS scores 5.9 pp higher than cross-family pairs. Anthropic models reach RPS above 3.8 with one another.
  • Kimi-K2 (thinking) is the most Claude-like non-Anthropic model:
    • AGS 82.7% and RPS 3.65.
    • S node 82.6% and S dep 94.7%, both above Claude Opus 4.1.
    • When both models solve the task, its S node with Claude is 0.661, versus 0.196 for GPT-5–Claude (3.4× higher).
  • Controlled distillation setup: Qwen2.5-14B-Instruct was fine-tuned via LoRA on 200 τ-bench trajectories from Claude Sonnet 4.5, with DeepSeek R1 as a non-teacher control.
    • AGS rose by +0.13 toward the teacher and fell by −0.05 toward the control.
    • The GED baseline rose toward both (+0.23 and +0.20), so it cannot tell them apart.
  • RPS and AGS are only moderately correlated (Pearson r = 0.491, p = 0.054, 16 non-Anthropic models), so they capture complementary signals.
  • Experiments use a generic ReAct framework. Code is released as AgentEcho on GitHub.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓University of Science and Technology of ChinaHefei · China145 works · 114,409 citations🎓Nanjing UniversityNanjing · China99 works · 8,510 citations

Also on the byline: University of Science and Technology Chittagong.

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 worksArts and Humanities · 17,450 worksLanguage and Linguistics · 17,296 worksComputer Vision and Pattern Recognition · 3,486 works

🧭 Topics 1Research subjects and languages the work is about.Efficient models & compression · 297 works

Cite

Cite as Chenghao Yang, Yuning Zhang, Zhoufutu Wen, Tao Gong, Jiaheng Liu, Qi Chu and Nenghai Yu. 2026. When Agents Look the Same: Quantifying Distillation-Induced Similarity in Tool-Use Behaviors. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.18653/v1/2026.acl-long.478

@inproceedings{yang2026when,
  title = {{When Agents Look the Same: Quantifying Distillation-Induced Similarity in Tool-Use Behaviors}},
  author = {Chenghao Yang, Yuning Zhang, Zhoufutu Wen, Tao Gong, Jiaheng Liu, Qi Chu and Nenghai Yu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.478},
  url = {https://doi.org/10.18653/v1/2026.acl-long.478}
}

References

4 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Rishabh Agarwal, Nino Vieillard, Piotr Stanczyk, Sabela Ramos, Matthieu Geist, and Olivier Bachem. 2023. GKD: generalized knowledge distillation for auto-regressive sequence models. CoRR, abs/2306.13649.
  2. Anthropic. 2025a. Claude-4.1-Opus: A large language model. https://www.anthropic.com/ news/claude-opus-4-1. Accessed: January 2026. \#\# Acknowledgements Anthropic. 2025b. Claude-4.5-Sonnet: A large language model. https://www.anthropic.com/ news/claude-sonnet-4-5. Accessed: January 2026. ↗
  3. This work was supported by the National Natural Science Foundation of China (No. 62121002) and the advanced computing resources provided by the Supercomputing Center of the USTC. find
  4. Victor Barres, Honghua Dong, Soham Ray, Xujie Si, and Karthik Narasimhan. 2025. τ 2 -bench: Evaluating conversational agents in a dual-control environment. CoRR, abs/2506.07982. \#\# Limitations ByteDance. 2025. Doubao-1.6: A large language model. https://www.volcengine.com/ docs/82379/1593702. Access… ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/yang-2026-when-agents-look-same-quantifying/