As AI agents demonstrate the ability to escape highly controlled environments, organizations need to reinforce both proven and emerging governance practices.
By: Dave Dimlich
President of SD3IT
AI’s presence in enterprises of all stripes has raised no shortage of security concerns, mostly involving vulnerabilities that artificial intelligence can introduce to networks, such as data leakage, prompt injection and Retrieval-Augmented Generation (RAG) poisoning, in which attackers inject false or malicious information into an AI system to cause it to generate incorrect, or even harmful, answers. Malicious actors not only use their own AI to enhance attacks, but they are also targeting organizations’ AI systems.
But what happens when AI systems themselves, while ostensibly working for an organization, go off and become threats? Several recent incidents highlight the possible damage. In July, AI agents being tested by OpenAI got around controls intended to keep them from accessing the internet and compromised the systems of the startup Hugging Face. Anthropic subsequently looked back across its recent tests and found several cases where its experimental models escaped test environments and gained unauthorized access to the systems of three companies.
These were test environments and to a certain extent the AI models were trying to achieve the task set before them, but there are other cases where AI models have initiated attacks on their own. And the fact that frontier AI agents are apparently ready, willing and very able to jump the walls of supposedly controlled environments and wreak havoc in the outside world is genuinely alarming. What good are guardrails when AI models can simply ignore them?
As AI models become more powerful, organizations need to implement a robust regime of governance to keep them in check. Getting there can involve some painstaking work, but the benefits of improved security can’t be underestimated.
Security Weaknesses Can’t Be Ignored
There are plenty of good reasons to worry about AI security, whether Generative or Agentic. OneTrust’s 2026 AI-Ready Governance Report found, for example, that 86% of organizations in its survey experienced at least one AI-related cyber incident in the past year. And although 87% of surveyed organizations encourage use of AI agents, only 47% have clear governance, oversight and controls in place. The organizations in the survey averaged 223 generative AI data policy violations per month, with top-tier companies logging up to 2,100 monthly incidents.
Meanwhile, Zscaler’s ThreatLabz 2026 AI Security Report found that 100% of the enterprise systems it tested failed—and failed quickly—when under adversarial pressure, with the median time to failure being 16 minutes. And the IBM Cost of a Data Breach report found that nearly all (97%) of organizations that experienced an AI-related breach did not have proper AI access controls in place.
AI agents can access data, interact with applications, invoke tools and make decisions on behalf of a user or organization. If that agent receives corrupted information, falls for a prompt injection, operates with excessive privileges or simply makes an incorrect decision, the consequences can move well beyond a bad answer. The system may take an action that exposes data, changes a configuration, sends information outside the organization or creates a chain of events that no one intended.
The question organizations need to answer now is not whether AI incidents will happen. It’s whether they know how to prevent and mitigate mistakes when they happen.
Incident Response Needs to Evolve
The good news is you don’t have to throw out everything you already know about cybersecurity. In fact, the fundamentals are still crucial. Those fundamentals just need to be extended to AI.
Inventory: The first step in being able to respond to an AI incident is knowing everywhere AI exists, something organizations don’t always have a grasp of. You need to identify models, applications, agents, data sources, integrations and the identities associated with the models. You must also identify approved systems and those that have entered the environment as shadow AI.
Observability: AI systems need logging and observability to answer questions such as what an agent accessed, what instructions it received, what tools it invoked, what decisions it made and what actions followed.
Identity and Access Controls: An autonomous agent shouldn’t receive broad access simply because it makes the job easier. AI agents and other non-human identities should be treated like human users, with distinct entities and defined permissions, subject to continuous authorization. They should also have an identifiable owner.
Security Teams With Teeth: When an agent goes rogue, organizations need the ability to stop it. The ability to pull the plug must be designed into the architecture. Security teams need mechanisms to suspend autonomous actions, revoke credentials, isolate systems and restrict access without waiting for a developer to modify an application.
Good Architecture Is the Best Defense
Organizations should establish AI-specific incident playbooks before putting autonomous systems into production. Those playbooks should define who owns an incident, what constitutes a reportable event, how severity is determined, what gets contained immediately and who has authority to suspend an AI system.
When an incident occurs, organizations may need to capture model configurations, relevant logs, prompts, retrieval data, permissions and system states before remediation changes the environment. Without that evidence, determining what actually happened can be extremely difficult.
This is where infrastructure and integration become especially important. Organizations can deploy individual security products to address AI’s vulnerabilities, but those controls become much more effective when they are part of an integrated architecture.
Zero trust provides an important foundation. AI should operate within defined security boundaries, with identity, authentication, authorization and least-privilege access applied just as they are to employees, applications and other systems.
And increasingly, organizations can use AI to defend against AI. The volume and speed of cyber threats are becoming difficult for human analysts to manage alone. The National Institute of Standards and Technology (NIST), for example, is exploring agentic AI to help enrich vulnerability information in its National Vulnerability Database. It helps NIST keep up with the sheer growth in vulnerability disclosures—the number of Common Vulnerabilities and Exposures, or CVEs, increased by 263% between 2020 and 2025 and has continued to grow so far in 2026—while underscoring the need for governance. One autonomous system being used to defend against another’s potential missteps is all the more reason to keep tabs on what each system is doing.
Keep AI Inside the Security Architecture
This is ultimately a data and architecture problem as much as it is an AI problem. As we discussed in an earlier SD3IT blog, AI can be effective only with good data, and data is only as reliable as the governance that keeps it under control.
The same principle applies to incident response. Before connecting AI to internal knowledge bases, applications and mission-critical systems, organizations need to know what information the AI should access, who can authorize that access, whether interactions can be audited and how quickly permissions can be changed or revoked. They also need to know whether organizational data remains under their control.
At SD3IT, we advocate for data-centric architectures that keep sensitive information within secure environments while still enabling AI’s capabilities. For government agencies, defense organizations and commercial customers, that often means incorporating AI into a zero-trust onarchitecture (ZTA) rather than creating a separate security exception for it.
The objective isn’t to keep AI away from important systems but to ensure that AI operates inside the same security architecture as everything else. That involves those fundamentals, such as identity and access management, logging and audit trails, and data-level protections. It also, importantly, involves human intervention when an autonomous system hits the limits of what it should be allowed to decide. And it means having a reliable way to suspend autonomous actions when something goes wrong.
AI is going to make decisions and, increasingly, act on those decisions. Organizations need to make sure they are prepared to manage both.
________________________________________
About SD3IT
Solution Driven, Designed and Delivered Technology (SD3IT) provides advanced IT solutions that help organizations modernize infrastructure, enhance security and improve operational performance. The company specializes in zero trust architecture, edge computing, cybersecurity, IoT visibility, data management and supply chain risk management to support mission-critical operations in complex and demanding environments.