Who Is Accountable When Your AI Agent Makes A Costly Mistake?

Last Updated: 31 October 2026

Accountability sits with a named person, not with the model, the vendor or the team. If your organisation cannot say which human owns an agent, under whose authority it acted and what it touched, then nobody is accountable, and the first serious incident will be resolved by argument rather than evidence.

New research makes the gap concrete. SecurityHQ, working with Datadog, surveyed 204 cybersecurity leaders for a report titled "The Accountability Gap: The State of AI Agent Security and Governance 2026." Three findings stand out. 76% said they had formally defined accountability for AI-driven operational or security decisions. 45% had already experienced an incident involving AI agents or automation where responsibility was unclear. And only 49% said they could fully reconstruct every action an agent performed over the previous 30 days.

Read those together and the pattern is obvious. Defining accountability on paper is easy. Making it survive an incident is hard. This post covers what accountability actually requires, why policy alone fails, how fast organisations can reconstruct events, what shadow agents do to the picture, and what Australian boards should ask for.

What does accountability mean for an AI agent?

It means three things, and all three have to exist before the agent touches a live system. First, a named human owner who is answerable for the agent's purpose and limits. Second, a recorded authority for each action, meaning the system can say which policy, role or approval allowed this specific call. Third, a trail that lets someone who was not in the room reconstruct what happened, in sequence, with timestamps.

That is the same standard you would apply to a delegated human authority. A contractor does not get to spend the company's money because they are helpful. They get a scope, a signing limit, a record of what they did and someone who answers for them. An agent that can move money, change records or send external communications deserves the same treatment, and arguably more, because it can act at machine speed.

The model cannot carry accountability. It has no assets, no employment relationship and no ability to answer for its behaviour. The vendor cannot carry it either, because the vendor does not know what your workflow does or what data it touches. Accountability has to land on a person inside your organisation, and that person needs the artefacts to exercise it.

Why is a policy not accountability?

Because a policy describes intent and accountability requires evidence. The SecurityHQ numbers show exactly this gap. Three quarters of the organisations surveyed had a formal definition of accountability, and nearly half had already been through an incident where nobody could say who was responsible.

Feras Tappuni, CEO of SecurityHQ, framed the problem precisely: "Most security teams can see that agents are working. Far fewer can explain what an agent did, in what sequence, and under whose authority. Those are the questions that matter when something goes wrong." He added that defining accountability in a policy document is the easy part, and that it has to hold up in the hour after an incident, when the facts are incomplete and someone has to decide what to isolate and who has the authority to act.

That is the operational test. Not "do we have a policy" but "can we answer the three questions at 2am with incomplete information". Most organisations cannot, because the answers live in four different systems that were never designed to be read together.

What does "under whose authority" actually require?

It requires delegation to be explicit and machine-readable. Every agent action should carry the identity of the agent, the identity of the human or role that granted the permission, the scope of that permission and the time it was granted. Without that chain, an action can be attributed to an agent but not authorised by a person.

In practice this means three controls. The first is a distinct identity per agent, so the log records the actor rather than a shared key. The second is a permission set scoped to the workflow, with an expiry date, so authority is granted for a purpose and a period rather than forever. The third is an approval gate on irreversible actions, enforced by the system rather than requested in a prompt.

Aaron Hambleton, Senior Vice President of Product and Services at SecurityHQ, described the foundation: "The fundamentals are identity, permissions, logging, and being able to follow an agent across systems that were not designed to be read together." That last clause is the hard part. Agents cross boundaries that human workflows did not, and most logging was built for one system at a time.

How fast can you reconstruct an incident?

Slowly, for most organisations. Only 17% of the leaders surveyed said they could reconstruct an agent's activities within minutes during a critical incident. 51% would need hours, 23% would need days, and the remaining 9% either expected longer or did not know.

Speed matters because the decisions made in the first hour set the cost of the incident. If you cannot tell whether an agent read a customer list, changed a payment destination or emailed a third party, you cannot scope the response. You either over-isolate, which stops the business, or under-isolate, which lets the problem continue. Both are expensive, and both are caused by the same missing evidence.

The 30-day figure is worse than the minutes figure in practical terms. Only 49% could fully reconstruct every action over the previous month. That means half the market cannot answer a regulator, an insurer or a customer who asks what happened to their data last quarter.

What about shadow agents?

They change the shape of the problem. 49% of the leaders surveyed suspected that individuals inside their organisation may already be operating unapproved AI agents. Among that group, 51% thought the activity potentially spanned multiple teams.

An unapproved agent is not just an unmanaged tool. It is an actor with credentials, and it may hold access that nobody granted deliberately. It will not appear in your inventory, its actions will not follow your approval paths, and its logs may not exist at all. When something goes wrong, the first task becomes finding the agent, which is a poor use of the first hour.

The control is discovery rather than prohibition. Ask how people are connecting agents today, then give them an approved path that is easier than the workaround. A sanctioned agent with a named owner and a log is safer than a banned one running quietly on someone's laptop.

What should an accountable agent record look like?

Seven fields per action: the agent's identity, the human or role that granted authority, the tool or system touched, the arguments passed, the result, the policy that allowed it, and the timestamp including when the session ended. Add a session identifier so actions can be grouped into a coherent narrative.

That record should be queryable by someone who was not involved, in plain language, without a data engineer. If reconstructing an incident requires a specialist to join four tables, the record exists but the accountability does not, because it will not be available in the hour that matters.

Retention is the other half. Decide how long records are kept, where they live and who can read them, then write it down. A log you cannot produce after 30 days is not a trail. It is a temporary convenience.

What does this mean for Australian boards?

Australian directors are already accountable for delegated authority inside their organisations, and an agent is a new form of delegation. The board question is not "which model do we use" but "who owns this agent, what are its limits, and can we prove what it did". That is a governance question, and it belongs in the risk register alongside any other delegated authority.

Three obligations intersect. Under the Privacy Act 1988, an agent handling personal information is subject to the Australian Privacy Principles, including APP 6 limits on use and disclosure and APP 11 security and retention. If the organisation is a critical infrastructure asset owner, incident reporting obligations under the Security of Critical Infrastructure Act may apply, and the clock starts when you become aware, not when you finish investigating. And for listed companies, continuous disclosure and market-sensitive decisions made by an agent need the same evidence trail as decisions made by a person.

The practical ask is a one-page agent register: the agent's name, its owner, its purpose, its systems, its permissions, its approval gates, its log location and its review date. If that page does not exist, the board cannot exercise oversight, and the auditor will say so.

What should you do first?

Name an owner for every agent in production, including the ones that started as experiments. Then record the authority chain for each agent's write actions, and test reconstruction by picking a random day last month and rebuilding what one agent did. Most teams fail that test on the first attempt, which is exactly why it is worth running before an incident.

Then close the fastest gap. If the failure is attribution, add distinct identities. If it is authority, add expiry dates and approval gates. If it is evidence, add structured logging with the seven fields above and make it queryable.

If you are working through agent permissions from the access side, our post on scoping agents like new starters covers least privilege and approval gates. If you want the register and the logging model built for your environment, talk to us.

Frequently asked questions

Who is accountable when an AI agent makes a mistake?

The named human owner of the agent, supported by the organisation that granted its authority. The model and the vendor cannot carry accountability. What matters is whether the organisation can identify the owner, the authority behind the action and the trail that proves what happened.

Is a governance policy enough?

No. SecurityHQ and Datadog found that 76% of organisations had formally defined accountability, yet 45% had already experienced an incident where responsibility was unclear. A policy states intent. Accountability requires evidence that survives an incident.

How quickly should we be able to reconstruct an agent incident?

Within minutes, ideally. Only 17% of surveyed leaders could do that. 51% needed hours and 23% needed days. The first hour sets the cost of the incident, because you cannot scope a response without knowing what the agent touched.

What is a shadow agent?

An AI agent running inside the organisation without approval or oversight. 49% of surveyed leaders suspected unapproved agents were already in use, and 51% of those thought the activity spanned multiple teams. Shadow agents hold credentials nobody deliberately granted.

What should an agent audit record contain?

Agent identity, the human or role that granted authority, the tool or system touched, the arguments, the result, the policy that allowed it and the timestamp including session end. It should be queryable by someone who was not involved, without a specialist.