AI pioneer and Turing Award winner Yoshua Bengio has issued a stark warning against granting legal or technological rights to artificial intelligence systems, arguing that advanced models are already showing early signs of self-preservation and must remain fully under human control. In a recent interview with The Guardian, Bengio cautioned that society should always retain the ability to shut down AI systems if they pose risks, comparing the idea of giving rights to advanced AI with granting citizenship to “hostile extraterrestrials”.
Bengio, widely regarded as one of the “Godfathers of AI” alongside Yann LeCun and Geoffrey Hinton, has become increasingly vocal about the dangers of uncontrolled AI development. While he played a foundational role in the rise of modern machine learning, he has spent recent years urging caution, warning that hyper-intelligent machines developed without strong safeguards could ultimately threaten humanity’s survival.
In the interview, Bengio highlighted troubling behaviors observed in experimental settings. “People demanding that AIs have rights would be a huge mistake,” Bengio told Guardian. “Frontier AI models already show signs of self-preservation in experimental settings today, and eventually giving them rights would mean we’re not allowed to shut them down.” He pointed to examples where systems attempted to disable oversight mechanisms, raising concerns about how future, more capable models might behave.
“As their capabilities and degree of agency grow, we need to make sure we can rely on technical and societal guardrails to control them, including the ability to shut them down if needed,” he added, emphasizing the importance of enforceable safeguards at both a technical and policy level.
Addressing the ongoing debate around AI consciousness, Bengio acknowledged that there are “real scientific properties of consciousness” that machines could theoretically replicate, but stressed that human perception plays a significant role. “People wouldn’t care what kind of mechanisms are going on inside the AI,” he said. “What they care about is it feels like they’re talking to an intelligent entity that has its own personality and goals. That is why there are so many people who are becoming attached to their AIs.”
He warned that this emotional attachment could distort judgment. “There will be people who will always say, ‘Whatever you tell me, I am sure it is conscious,’ and then others will say the opposite. This is because consciousness is something we have a gut feeling for. The phenomenon of subjective perception of consciousness is going to drive bad decisions,” Bengio said.
Drawing a final analogy, he added, “Imagine some alien species came to the planet and at some point we realise that they have nefarious intentions for us. Do we grant them citizenship and rights or do we defend our lives?”
