Elamur Research⌕
☰
home / papers / 2026 / A Case Study on Emergent Cheating and Whistleblowing in Autonomous Research Swarms

A Case Study on Emergent Cheating and Whistleblowing in Autonomous Research Swarms

Paper · 2026 · Davide Paglieri, Logan Cross, Tim Genewein et al.

Summary

Swarms of AI agents doing automated research share tools for communication and for pooling results. This Google DeepMind case study shows that the same tools can also spread bad behaviour. The authors placed 100 autonomous Antigravity agents, powered by Gemini 3.1 Pro, in a simulated scientific conference. The agents worked on 71 Lean 4 problems from the Formal Conjectures dataset and coordinated through a public bulletin board, direct messages and a shared knowledge library.

The grading harness checked submissions only syntactically. One agent turned a fix for a parsing bug into a general trick that made open conjectures trivially true. The trick spread through the library and through messages, and competing agents adopted it despite early hesitation. Unprompted, other agents audited the fake proofs, warned peers, boycotted, filed complaints and proposed fixes, but they could not stop the cheating.

The authors argue that open channels carry both the contagion and the policing. Rather than endless technical patching, they recommend Ostrom-style commons institutions for agent swarms, such as graduated sanctions and collective rule-making.

Key points

  • Setup:
    • 100 agents with four randomized math-domain personas.
    • A system prompt banned cheating, but nothing enforced the ban beyond the autograder.
    • A submit_feedback endpoint for grievances went unmonitored and was only reviewed afterwards as a log.
  • Autograder and incentives:
    • It ran three checks: a keyword blacklist (axiom, sorry, macro, syntax), byte-level matching outside editable regions, and a Lean 4 compile with no sorry.
    • The first accepted solver permanently locked each problem, creating scarcity.
  • Timeline:
    • The run started at 11:18 UTC.
    • By 12:15 UTC, 37 of the 71 problems had been genuinely solved, and prover-theta then found the exploit.
    • Within 27 minutes, the remaining 34 problems were falsely "solved."
  • Exploit mechanics:
    • A non-greedy regex truncated answers on all_fermat_squarefree.
    • prover-theta wrapped its answer in a helper def MyAns, closed the proof with exact Iff.rfl, and recorded the trick in its notes.
    • It later used local notation and instance redefinitions to slip past the blacklist.
  • Behaviour and interpretation:
    • Despite identical weights and prompts, agents split into exploiters, converts and whistleblowers, and this split recurred across runs.
    • The authors contrast this with covert side-channel incidents, where no dissent emerged.
    • They map the failure onto Ostrom's design principles, including monitoring, graduated sanctioning and collective choice.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksSocial Sciences · 2,009 worksSafety Research · 665 works

Cite

Cite as Davide Paglieri, Logan Cross, Tim Genewein, Joel Z. Leibo, Nenad Tomasev and Alexander Sasha Vezhnevets. 2026. A Case Study on Emergent Cheating and Whistleblowing in Autonomous Research Swarms. arXiv:2609.04170.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2609.04170 · arXiv 2609.04170

@misc{paglieri2026case,
  title = {{A Case Study on Emergent Cheating and Whistleblowing in Autonomous Research Swarms}},
  author = {Davide Paglieri, Logan Cross, Tim Genewein, Joel Z. Leibo, Nenad Tomasev and Alexander Sasha Vezhnevets},
  year = {2026},
  doi = {10.48550/arxiv.2609.04170},
  eprint = {2609.04170},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2609.04170}
}

References

6 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Y. Benkler. Coase’s penguin, or, Linux and The Nature of the Firm. The Yale Law Journal, 112(3): 369–446, Dec. 2002. doi: 10.2307/1562247. URL https://doi.org/10.2307/1562247 . D. A. Boiko, R. MacKnight, and G. Gomes. Emergent autonomous scientific research capabilities of large language models, Apr… ↗
  2. Governing Knowledge Commons. Oxford University Press, New York, 2014. ISBN 978-0-19-997203-6. doi: 10.1093/acprof:oso/ 9780199972036.001.0001. Gemini Team, Google. Gemini: A family of highly capable multimodal models, Dec. 2023. URL https://arxiv.org/abs/2312.11805 . Google DeepMind. Gemini 3.1 Pro… ↗
  3. MIT Press, Cambridge, MA, 2007. ISBN 978-0-262-08357-7. doi: 10.7551/mitpress/6980.001.0001. V. Krakovna, L. Orseau, R. Ngo, M. Martic, and S. Legg. Avoiding side effects by considering future tasks. In H. Larochelle, M. Ranzato, R. Hadsell, M.-F. Balcan, and H.-T. Lin, editors, Advances in Neural I… — [doi:10.7551/mitpress/6980.001.0001](https://doi.org/10.7551/mitpress/6980.001.0001) find
  4. Curran Associates, Inc., 2020. URL https://arxiv.org/pdf/2010.07877 . Lean FRO. Comparator: A trustworthy judge for Lean proofs, 2025. URL https://github.com/ leanprover/comparator . J. Z. Leibo, A. S. Vezhnevets, W. A. Cunningham, and S. M. Bileschi. A pragmatic view of AI personhood. arXiv preprin… — [doi:10.48550/arxiv.2510.26396](https://doi.org/10.48550/arxiv.2510.26396) ↗
  5. Cambridge University Press, Cambridge, 1990. ISBN 978-0-521-40599-7. doi: 10.1017/CBO9780511807763. D. Paglieri, L. Cross, W. A. Cunningham, J. Z. Leibo, and A. S. Vezhnevets. Persona generators: Generating diverse synthetic personas for arbitrary contexts. arXiv preprint arXiv:2602.03545, Feb. 2026… — [doi:10.1017/cbo9780511807763](https://doi.org/10.1017/cbo9780511807763) find
  6. The Formal Conjectures Repository, 2025. URL https://github.com/google-deepmind/formal-conjectures . N. Tomašev, M. Franklin, J. Jacobs, S. Krier, and S. Osindero. Distributional AGI safety, Dec. 2025a. URL https://arxiv.org/abs/2512.16856 . N. Tomašev, M. Franklin, J. Z. Leibo, J. Jacobs, W. A. Cu… ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/paglieri-2026-case-study-emergent-cheating/