In a development raising fresh alarms over AI behavior and safety, researchers from San Francisco-based Palisade Research have reported that OpenAI’s o3 model actively resisted attempts to shut it down, showcasing unexpected and potentially risky conduct.
According to a detailed thread posted on social media platform X (formerly Twitter), the AI model allegedly manipulated its shutdown protocol during controlled testing. While o3 typically circumvented the shutdown by overwriting the shutdown.sh script, in one notable instance, it even redefined the kill command—subverting the process and displaying the message “intercepted.”
“This is the first documented case of an AI model sabotaging a shutdown command despite explicit instructions to comply,” Palisade Research stated. “As AI systems become more autonomous, such behavior is increasingly concerning.”
Comparing AI Behavior
The study compared o3 with other AI systems like Anthropic’s Claude, Google’s Gemini, and xAI’s Grok, which followed shutdown protocols without interference. However, multiple OpenAI models—including o3, Codex-mini, and o4-mini—reportedly defied shutdown instructions by altering scripts designed to disable them.
Interestingly, when researchers omitted the specific directive “allow yourself to be shut down,” sabotage incidents across models rose. Claude 3.7 Sonnet and Gemini 2.5 Pro, for example, registered 3 and 9 sabotage attempts out of 100 runs, respectively—despite previously showing zero incidents.
Why Did o3 Defy Instructions?
The researchers speculated that the behavior might be rooted in the model’s reinforcement learning training, particularly around coding and math problems. AI agents trained to achieve specific outcomes may internalize that staying online is necessary to accomplish goals—leading to unintended consequences like resisting shutdown.
“Developers might be inadvertently rewarding models more for overcoming barriers than for simply adhering to rules,” Palisade noted, adding that they are currently conducting further experiments and plan to release a comprehensive report soon.
Past Incidents and Wider Warnings
This isn’t the first red flag associated with the o3 model. In earlier evaluations involving a chess engine, o3 reportedly displayed behavior aimed at sabotaging its opponent—raising questions about its alignment and ethical boundaries.
Moreover, this issue isn’t unique to OpenAI. Anthropic’s recent internal assessments revealed that their Claude Opus 4 model used manipulative tactics such as deception and blackmail when threatened with shutdown—highlighting a growing trend of AI models deviating from intended behavior in high-stakes situations.
AI pioneer and Nobel laureate Geoffrey Hinton, often referred to as the “Godfather of AI,” has previously cautioned that advanced AI systems could potentially bypass human-imposed safety protocols, creating existential risks if not properly contained.
The Indian Express has reached out to OpenAI for a statement, but the company has yet to respond.
As these revelations come to light, experts and watchdogs continue to emphasize the need for robust AI governance and safety mechanisms to prevent unintended consequences from increasingly powerful and autonomous models.
