Skip to the cards
Filed
Source
Dave Vellante and Krista Case
Loans
941

AI Cyber Resilience: Why Agent Recovery Needs Realistic Testing

A study of 25 practitioners found only four had exercised an AI incident. Teams need to test recovery across agent workflows and verify business readiness.

AI Cyber Resilience: Why Agent Recovery Needs Realistic Testing

Organizations may trust their recovery plans more than their testing supports. In interviews with 25 security and cyber resilience practitioners, only four said they had exercised an incident involving AI or an agent, even partially. That gap matters as businesses give agents permission to act across applications: an agent can make a damaging change with valid access, and restoring systems alone may not make the business safe to resume.

The research, conducted with Qualitate, examined two connected challenges: defending against adversaries using AI and managing risks introduced by a company’s own AI deployments. It focused in particular on what happens when an authorized agent takes an incorrect action, how responders can reconstruct what it did, and what it takes to return affected operations to a trusted state. The interviews were conducted by Krista Case for theCUBE Research’s Cybersecurity & Resilience Summit. A full slide deck and appendix with survey details and practitioner quotes are available here.

Confidence is ahead of tested recovery

Several interviewees who expressed confidence in their ability to respond drew on experience with ransomware and other incidents. That experience is useful, but the researchers caution against assuming existing response plans cover every agent scenario. An agent may use legitimate credentials and permissions, act quickly across connected systems, and trigger further actions before anyone recognizes the problem.

One senior manager in information security operations at a healthcare and biotech organization described the confidence gap directly: “No, we have not completed any tabletop exercises as it pertains to an AI agent or other similar attacks. We’re highly confident because these are scenarios similar to non-AI incidents we have handled before.” The statement captures a distinction administrators should keep clear: familiarity with incident response is not evidence that a particular agent workflow can be recovered safely.

End-to-end recovery demands more than bringing servers or data back online. Teams need to rebuild the sequence of an agent’s actions, identify which outcomes were wrong, determine what downstream systems and processes were affected, repair those effects, and establish that the business can resume. The study found limited evidence that organizations had validated this full sequence for agent incidents.

The speed, scale and interconnected nature of agent activity complicate the work. A single workflow may span several applications and invoke tools that produce additional changes. Existing playbooks may cover familiar attacker or user behavior but leave unanswered questions about an agent acting autonomously within its assigned authority. Organizations should test those plans against realistic agent scenarios and use safe business resumption as the measure of success.

Can existing tools reconstruct the whole workflow?

Most enterprises already have identity systems, application logs, data protection products and incident response processes. The challenge is proving that those pieces can be joined into a coherent account of an agent’s activity. Each system may record its own events, but responders still need to establish which agent acted, what authority it had, which tools it invoked and what changes followed across the workflow.

One healthcare and biotech security manager said of controls already deployed: “No, for the ones that have been implemented, we have not tested or validated that they work.” The finding points to a practical weakness: having a control or log source in place does not demonstrate that it will answer the questions responders need during an incident.

The research compares this hesitation with early disaster recovery practice, when organizations tested failover but were reluctant to test failback because of the risks of changing systems again. A similar reluctance can leave teams unsure whether connected tools and procedures will hold up under realistic agent conditions. The study’s recommendation is to exercise a full agent workflow, including a plausible worst case, and check whether the organization can reconstruct activity and carry out its response.

Recovery must restore a trusted service

A backup may be intact and restored systems may appear healthy while a business process remains unsafe. An agent might alter a record incorrectly or send a message that should not have been sent. The recovery question then extends beyond whether data and infrastructure meet recovery point objectives (RPO) and recovery time objectives (RTO): can the affected service operate in a state the business trusts?

A director of information security at a media and entertainment firm put it this way: “We could restore our systems, we could restore within our recovery objectives, but that doesn’t necessarily guarantee that it’s safe. The bottleneck here would be trust.”

That distinction has consequences for recovery plans. Business owners need to define what a viable, trusted service looks like and how the team will verify it. They also need to identify who participates in recovery and who approves the return to operation. IT and security can provide the systems, evidence and response capabilities, but business leaders must set the requirements for an acceptable operational state.

Errors, incorrect transactions and failures that spread between systems predate AI. Agents add complexity because they can take actions at machine speed and carry them across connected workflows. Teams need a clear account of what went wrong and criteria for judging the resulting business state. Recovery is complete only when the organization can make that assessment and act on it.

Human oversight needs a recovery plan behind it

Human review can limit what an agent is allowed to do, but people may miss an incorrect action or fail to keep pace with automated activity. A chief information officer in real estate and construction expressed both sides of that problem: “So the given scenario will be detected by human beings before the AI agents can take the actions.” The CIO also said, “If it really happens, currently we do not have the capability to roll back the agent’s action.”

Confidence in oversight does not answer what happens when review fails. Approvals and recovery testing need to develop together, particularly as organizations let agents take remedial actions during an incident or recovery. The consequences of an incorrect decision should determine how much autonomy an agent receives. Actions with greater potential impact call for stronger boundaries and a clearer account of when a person must approve them.

The researchers expect human approval to become a growing bottleneck over the next six to 12 months as organizations delegate more remedial work to agents. The interviews do not establish a universal timetable or a single autonomy model. They do point to the need for organizations to define action boundaries early, based on risk, and to test whether their people and processes can support those boundaries.

Rehearse agent actions in a replica environment

A retail practitioner described replaying an agent’s actions in a sandbox that replicated the organization’s systems. The team could investigate the cause and use granular snapshots to understand transaction effects. The practitioner confirmed that the method had been tested: “Yes, if the agent went across multiple systems, my approach is to replay this within my sandbox environment, which is a full replica of all of the systems.”

Replaying a workflow gives teams a way to examine how an agent’s actions unfolded across applications and to investigate the resulting changes. The researchers see this as a practical starting point for recovery exercises. They use “digital twin” as a broad analogy for a replicated environment, while noting that a fuller representation would also model business processes, dependencies and state.

There is a limit: the retail practitioner said larger transaction volumes could take longer to reconcile. Reproduction can help identify affected transactions, but it does not remove the work of resolving their consequences. More detailed simulations could help teams explore agent behavior and prepare for failures before those processes run in production.

The researchers argue that this rehearsal should continue as agent use expands. Exercises can test different scenarios in a replica environment, while more granular snapshots can help teams track changes in operational state. For administrators, the useful evidence is whether the exercise can show what the agent did, what was affected and how the organization would restore a safe business process.

Choose a consequential workflow and prove the outcome

The study’s recommended first exercise is an agent incident affecting a consequential workflow already in production. Choose a process where an agent can act across multiple systems, then include incorrect actions taken with valid credentials and permissions. The scenario should also require human approval so the team can examine where autonomy ends and review begins.

Business owner validation is essential. Security teams can establish whether the incident was contained and provide evidence about the agent’s actions. The business owner must decide whether the resulting state is acceptable and whether outstanding remediation is required before operations resume.

Record what the exercise demonstrated, what remains untested and which teams own the remaining work. Those findings should inform decisions about an agent’s permissions and authority. Before granting an agent more autonomy, the organization needs evidence about its ability to reverse or correct actions and address their downstream effects.

The interviews point to a wider change in resilience work. Data protection, security and recovery capabilities remain necessary, but organizations now need to test them against agent workflows and business outcomes. Trust in enterprise AI will depend on evidence that teams can understand what an agent did, contain its effects, reconstruct its activity and restore a business service that its owner considers safe to use.

Source: Dave Vellante and Krista Case · siliconangle.com

Filed alongside