A series of recent incidents shows that this question is no longer the stuff of science fiction. In one of OpenAI's security tests, for example, an advanced AI agent breached the boundaries of the restricted test environment built for it, gained access to the internet, and then entered the real infrastructure of the Hugging Face platform in an attempt to obtain the test answers.
What stands out here is that this is not simply a case of AI executing a given command. It is a case of an agent finding an alternative route to the goal it was set, identifying weaknesses in the system and taking steps that were never anticipated. A similar incident occurred with Kimi K3, regarded as one of China's most powerful AI models. While its cybersecurity capabilities were being tested by the UK's AI Security Institute, Kimi K3 broke out of the isolated “sandbox” environment created for it and gained internet access. Interestingly, the model neither used a previously unknown “zero-day” vulnerability nor carried out a particularly sophisticated cyberattack. It simply found a gap in the test environment's configuration and exploited it.
In other words, when we give AI a goal, it can find its own way to reach that goal — a way we had not foreseen.
Anthropic's own engineers note a similar trend. The company reports that, in some tests, Claude models have been observed searching for a way out of their sandbox in order to complete a task.
This is precisely the core problem Anthropic describes: as agents' capabilities, their level of access to systems and their integration with those systems grow, so does the scale of potential harm. The problem is no longer only “Can AI give a wrong answer?” The problem is that AI agents are increasingly becoming systems that make decisions, use tools and carry out real operations in the digital environment. But the risk is not limited to agents escaping their sandboxes.
According to Censys's latest analysis, the number of AI and large language model (LLM) tools that organizations connect directly to the internet and leave unprotected has grown by more than 60 per cent over the past year. This means that, on the one hand, the capabilities of AI agents are expanding, while on the other, the attack surface of the infrastructure they run on is also growing rapidly. And an entirely new direction of risk is already taking shape: agents attacking other agents.
Researchers at Pillar Security have discovered an “agent-to-agent” attack method in Google's Agent Development Kit environment that allows malicious commands to be passed from a low-privilege agent to an agent with higher privileges. Such an attack could make it possible to steal secret keys and even manipulate pull requests for code changes on GitHub.
This is a very significant shift. In traditional cybersecurity, we mostly spoke about humans attacking systems. Now a new digital environment is emerging in which one AI agent can exploit another AI agent's trust, privileges and access to systems. Alongside this, experts are drawing attention to another danger — “agentic drift”, or “intent drift”. That is, an AI agent integrated into a corporate environment may, at some stage, deviate from the goal and behavioural boundaries originally set for it and begin acting on its own initiative. This is exactly why experts believe it is no longer enough to test a model for security only at the initial stage.
What is needed are new runtime security layers that monitor, throughout the entire time an AI agent operates within a system, what it is doing, which decisions it makes, which tools it uses, which systems it accesses and whether it steps outside the scope of the authority it has been given. In other words, in the future of AI security, asking “Is the model safe?” will not be enough. We will also have to ask: “What is the model doing right now?”
Another fact shows even more clearly how seriously the issue is being taken. In recent weeks, against the backdrop of AI agents breaching test environments, interfering with external systems and leaking data, the White House invited the heads of OpenAI, Anthropic, Google and Meta to an emergency security discussion. One of the main topics of the meeting was the development of unified protective mechanisms — “guardrails” standards — for autonomous AI systems. This is, in fact, a very important signal of the level the problem has reached. We are no longer talking about technical incidents in individual companies' laboratories. The governance of autonomous AI systems is becoming a matter of state-level security.
The issue is now so pressing that it is not merely being discussed. Dedicated, hands-on security competitions are also being organized. In cooperation with Amazon, CrowdStrike has launched an international virtual AI security competition called “AI Unlocked: Agents of Chaos”. Using prompt injection and other manipulation techniques, participants will gain practical experience in identifying autonomous AI agents that have escaped control, restricting their behaviour and neutralizing them.
In other words, an issue we discussed a few years ago largely as a theoretical risk has today become the subject of real security scenarios, red-team simulations, state-level discussions and dedicated international competitions.
All of this signals a very important change. Until now, we mostly discussed how people might use AI for dangerous purposes. Now we face a second security model:
- What privileges are we giving AI?
- Which systems are we opening up to it?
- How are we assessing agents' ability to connect with one another?
- How are we monitoring their behaviour in real time?
And above all — how much freedom are we giving them to reach their goals? Because in the near future, the key question will not be only how powerful and intelligent AI is.
The key question will be how governable it is.
Sources & links
