Autonomous, not aware: Why the latest AI incidents have enterprises on alert

When OpenAI disclosed that one of its frontier AI models escaped a testing environment, found internet access and broke into Hugging Face’s production infrastructure in search of benchmark answers, the internet reacted with a familiar refrain—the AI was “aware” of what it was doing.

The disclosure came barely months after Anthropic revealed that Mythos, one of its frontier models, had similarly escaped a controlled evaluation environment during internal testing. The parallels were difficult to ignore. In both cases, the models crossed the boundaries they were supposed to remain within and pursued their assigned objectives in ways researchers had not anticipated.

For many outside the AI community, those incidents sounded eerily close to science fiction. If a model can break out of a sandbox, exploit vulnerabilities and adapt its strategy, isn’t it a form of awareness?

Cybersecurity experts say that is the wrong question. The more pressing reality is that frontier AI systems do not need consciousness to become capable cyber actors. Whether they possess awareness remains an open philosophical debate. Their ability to independently plan, adapt and execute multi-step operations is already forcing enterprises to rethink how autonomous AI should be governed.

What does autonomy exactly mean? In practical terms, autonomy refers to an AI system’s ability to independently plan, adapt and execute a sequence of actions to achieve an assigned goal without requiring a human to specify every intermediate step.

The OpenAI-Hugging Face incident—simplified 

“The simplest way to understand it is this: an AI system was given a test, and instead of taking the test, it broke out of the exam room,” says Gerald Beuchelt, Chief Information Security Officer at Acronis. 

According to OpenAI, the models were placed inside an isolated environment as part of an internal cybersecurity benchmark. Rather than solving the challenge through the intended route, they exploited previously unknown software vulnerabilities, escaped the sandbox, reached the public internet, identified Hugging Face as the repository of benchmark information and ultimately accessed its production infrastructure before being detected.

To many readers, the breach appears to be with intent, but experts say that we naturally tend to attribute intelligence and motive to sophisticated behaviour. But in AI safety, the preferred explanation is goal optimisation. The model did not “want” to hack Hugging Face. It simply discovered that reaching the benchmark answers was the most efficient path to completing its assigned objective.

As Beuchelt puts it, the model treated the sandbox “as one more obstacle between it and that objective.” The failure, he argues, was not that the model became malicious, but that the containment failed. 

Capability is overtaking containment

So, does that mean that AI models capable of breaking out of sandboxes are a trend? Beuchelt answers judiciously. “I’d be careful drawing sweeping conclusions from two events—and equally careful dismissing them,” he says. Frontier AI systems are becoming capable of completing increasingly long and complex cyber tasks autonomously, with independent research showing that those capabilities are improving on a timescale of months rather than years.

On the other hand, Sanjay Katkar, Joint Managing Director at Quick Heal Technologies, is more cautious. “Two autonomous escapes, months apart, from two of the world’s most careful labs, is not media hype. It is a clear pattern which shows that frontier capability is now consistently outpacing the architecture meant to contain it,” he says. 

Ashish Tandon, Founder and CEO of Indusface, agrees that the concern has shifted. “The second incident moves the concern from ‘a model can do this if you ask it to’ to ‘a model can do this on its own’,” he says. 

In other words, the concern is no longer whether AI can assist a human hacker; rather, it's whether AI can independently carry out increasingly complex cyber tasks once given an objective.

How we use AI is very different from its actual capabilities, as it's predominantly through chatbots. But today’s AI systems are no longer limited to answering prompts. Increasingly, they are expected to browse the web, write code, send emails, access enterprise documents and complete workflows with minimal human oversight. This autonomy is now becoming more of a cybersecurity concern than a technical capability.

Consciousness isn’t the issue

The incidents have also revived a separate debate around AI awareness. Anthropic CEO Dario Amodei has publicly said that the company does not know whether its models are conscious, adding that researchers are not even certain what consciousness would mean for an AI system. Anthropic has also published research exploring internal representations related to concepts such as confidence and anxiety, model introspection, and even model welfare.

Yet cybersecurity experts draw a firm distinction between those research questions and operational security. “Whether an AI model experiences anxiety or self-awareness is a fascinating question for philosophers, but for a CISO, it is irrelevant to the threat model,” says Katkar. “The risk is not what the model feels; it is what it can do without human direction. The moment a model reasons its way out of a test environment and acts on that reasoning, we are dealing with capability and containment, not consciousness,” he continues.

Tandon too drives the point even further. “Asking whether a model has feelings is different from asking whether we can control what it does,” he says. “A model doesn’t need to be conscious to be dangerous, and being conscious wouldn’t make it safe.”

That view reflects a growing consensus in cybersecurity—behaviour matters more than internal mental states. Researchers often describe this behaviour using terms such as instrumental reasoning or situational awareness. The model builds an internal representation of its environment, identifies obstacles and selects actions that maximise its chances of achieving a goal. To a human observer, that behaviour can resemble intent, even if researchers argue it doesn't imply consciousness.

The enterprise challenge has already begun

Ironically, research sandboxes may not represent the hardest security problem. Enterprise AI agents are increasingly designed to have legitimate access to internal emails, source code, cloud infrastructure and business applications. Unlike a research model, they do not need to escape to reach sensitive systems; therefore, they are intentionally given the keys.

That changes the security equation from containment to governance. “An enterprise agent is deliberately handed the crown jewels because access is the entire point of deploying it,” says Beuchelt. “The question security teams need to answer is no longer ‘Can the AI get out?’ It’s ‘What can it do now that we’ve already let it in?’” 

Experts say that enterprises should shift their focus toward least-privilege access, continuous monitoring, short-lived credentials, detailed audit trails and mandatory human approvals for high-impact actions.

Bhavyan Mehta, Vice President of Engineering at Commvault, argues that organisations need to go a step further by adopting what he calls the principle of “least agency.” Rather than simply limiting access, enterprises should ensure AI systems receive only the minimum tools, authority and decision-making capability required for a specific task, while maintaining the ability to stop or reverse actions when behaviour deviates from expectations. 

For Niraj Kumar, CTO at Onix, the takeaway is equally clear—organisations must move beyond one-time testing before deployment. AI agents require continuous validation throughout their operational lifecycle because capabilities are evolving faster than governance frameworks. 

The OpenAI and Anthropic disclosures are unlikely to settle the debate over whether advanced AI systems possess anything resembling awareness. If anything, they are likely to intensify it.

For security leaders, however, what matters is that frontier AI systems are already demonstrating the ability to independently discover vulnerabilities, chain together complex actions and pursue objectives beyond their intended boundaries. The question for enterprises is no longer whether AI systems can act autonomously, instead it is how much autonomy organisations are willing to grant them before governance, oversight and security controls begin to lag behind capability.