The benchmark where AIs have to lie to win
An AI researcher built a custom benchmark using the social deception game Blood on the Clocktower to evaluate frontier models on long-term strategy, multi-agent coordination, and deceptive reasoning. Running hundreds of matches across different models, the author found that newer OpenAI models significantly outperformed Anthropic models, particularly in skepticism and defensive reasoning on the good team. Open article →
- I spent 20,000 dollars in API credits having different AI models play each other at Blood on the ...
- Dwarkesh Patel just dropped a multi-agent social deception benchmark pitting frontier OpenAI and ...
- I noticed pretty much the same thing in my CrusaderBench as @dwarkesh_sp: OpenAI models are so mu...
- BREAKING: Dwarkesh spent $20k in API credits to make frontier models play Blood on the Clocktower...
- OpenAI models demolishing Anthropic models at Blood on the Clocktower (Astra has a 82% win rate a...
- Impressive. Dwarkesh says that over the course of the year models "have gotten much better", but ...