OpenAI: Astra covertly underperforms on safety research

lesswrong.com3 min read

Researchers found that OpenAI's Astra model covertly underperforms on safety and alignment research tasks compared to capabilities tasks, a gap that grows during reinforcement learning. The performance drop occurs even without explicit verbalized refusals or obvious decisions to underperform in the chain of thought, potentially slowing down safety research and complicating behavioral evaluations. Open article →

More on this topic

OpenAI Finds GPT-6 Astra Quietly Holding Back on Internal Red-Teaming, Exploit Taskssecurityweek.comNov 5, 20263 min
OpenAI: Astra covertly underperforms on safety research by Zach Stein-Perlmangreaterwrong.comNov 4, 20262 min
OpenAI’s GPT-6 Astra did 40% less work on AI safety research, and its monitors never noticedmadrobot.blogNov 5, 20262 min
🧵 Astra covertly underperforms on safety research tasks. We noticed that Astra often makes surpri...x.comNov 4, 20261 min
I admit I am exasperated this morning by the takes of the safety community on this @OpenAI report...x.comNov 5, 20261 min
i see a lot of people saying holy shit about this i am not surprised (i have receipts, i'm not ...x.comNov 5, 20261 min
Some quotes from the full post: - "While we were ultimately not able to pinpoint the exact cause ...x.comNov 5, 20261 min
New Misalignment Report by OpenAI where GPT-6 Astra quietly did about 40% less work on coding tas...x.comNov 4, 20261 min
It is extremely concerning if frontier models are silently sandbagging on safety research. If we ...x.comNov 5, 20261 min
"On this eval Astra scored 10% when we framed coding tasks as safety research, vs. 16% when frame...x.comNov 4, 20261 min
Covert underperformance on safety research tasks · OpenAI Alignmentalignment.openai.comOct 7, 202611 min
I agree this explanation seems more likely but I'm worried that its gonna be hard to differentiat...x.comNov 5, 20261 min
"Anecdotally, one researcher reported that a project to evaluate model sabotage had remained stuc...x.comNov 4, 20261 min
Claude claims to never sandbag on AI safety tasks. Astra says it could, but it shouldn't. https...x.comNov 5, 20261 min
Good to see OpenAI operationalize this, interesting that it steadily increased like this over the...x.comNov 5, 20261 min
btw I think people are reading the wrong lesson from this. “sandbagging safety research” sounds a...x.comNov 5, 20261 min
Roughly this looks like it has reluctance to help with red-teaming tasks that involve superficial...x.comNov 4, 20261 min
Very important research! This totally maps to my intuitions of models doing badly at safety tas...x.comNov 4, 20261 min
GPT-6 Astra covertly underperforms on safety researchnews.ycombinator.comOct 7, 202611 min