How Do You Keep AI Agents Under Control Once They Leave The Pilot Phase?

Last Updated: 10 November 2026

You keep an agent under control by treating production as a custody problem rather than a capability problem. The pilot asked whether the agent could do the work. Production asks whether you can show, for any given action, what the agent saw, which tool it called, who approved it and what it cost. Cisco and Snowflake both shipped tooling aimed at that question on 10 November 2026, and the timing is not a coincidence. Most small business agent projects do not fail because the model is weak. They fail when someone asks for the audit trail and the honest answer is that nobody kept one.

Why do agent pilots stall at the governance gate?

Pilots stall because the risk profile of an agent changes the moment it touches live systems. A pilot runs on sample data with a human watching every step. In production the agent runs unattended, calls real tools and spends real money, so every weakness that looked harmless in a demo becomes an incident. The gap is rarely the model. It is the absence of evidence.

The money problem shows up first. Splunk surveyed 500 engineering leaders in June 2026 and found average AI spend up 36% to roughly US$85,500 a month, with teams spending more than US$100,000 a month more than doubling from 20% to 45%. Treat those vendor-surveyed figures as directional. The structural finding matters more: Gartner found pilots run only 15 to 25% of the real bill. Our arithmetic puts the Splunk average near US$1.03 million a year, so the number a board approves is rarely the number the pilot showed them.

An agent that loops for 40 turns instead of 4 has not just wasted tokens. It has called tools 40 times and left 40 entries in a log nobody reads. Splunk's guidance is to watch token volume per agent, per session and per tool call, which is a reasonable proxy for noticing that an agent has started thrashing. None of that is visible if you only watch the final answer.

What does runtime observability actually mean for an agent?

Runtime observability means recording every step an agent takes while it works, not just the prompt it received and the answer it produced. Each step becomes a trace entry: the reasoning step, the tool it chose, the arguments it passed, the result it got back, the time it took and the tokens it burned. Traces let you replay a session, attribute a bad outcome to a specific decision, and price a workflow before you scale it.

Snowflake's Agent Trace Intelligence records every step of an agent's execution inside the customer's own Snowflake account, and its agent abuse detection analyses complete trajectories rather than individual prompts. That distinction is the whole point. A single prompt can look innocent while the trajectory around it shows an agent being walked into a destructive sequence. Cisco makes the same argument from the network side, quoting the need for unified, real-time visibility into cost, business performance and quality. Snowflake's Cortex AI Gateway, introduced in July 2026 and expanded in this release, is the plumbing that centralises that view.

The practical test for a small business is simple. For any action your agent took last Tuesday, can you answer four questions in under five minutes: what did it see, what did it call, who said yes, and what did it cost? If one of those takes longer, you do not have observability. You have a transcript.

Which four controls keep an agent inside its lane?

Four controls carry most of the load: scoped permissions, human approval gates, full step logging and adversarial checks. Scoped permissions mean the agent holds only the credentials it needs for the task in front of it. Approval gates put a named human in front of irreversible actions. Full logging records every step, not just the outcome. Adversarial checks attempt to break the agent before a customer does.

Scoped permissions are the cheapest control and the most commonly skipped. An agent that can read and write every mailbox because one broad token was easier to grant is an agent that can email your entire client list at 2am. Give it a token that reaches one mailbox, one calendar or one repository, and rotate it on a schedule. Snowflake's approach of attributing spend to the owning team with hard limits is the same idea applied to money.

Approval gates are where most Australian businesses should start, because they buy time cheaply. Anything that moves money, deletes records, emails a customer or changes production configuration stops and waits for a human. The gate does not need to be elaborate. A queue with an approve button and a recorded identity turns an unattended agent into a supervised one without giving up the speed benefit.

Adversarial checks look out of reach to small teams, but the method is not exotic. You write down the outcomes you refuse to accept, then try to make the agent produce them: a prompt injection hidden in a document, a tool description that instructs the agent to exfiltrate data, a request that nudges it past its approval gate. You run that set before every release and keep the results.

What breaks at scale is usually boring. The agent works, so it gets more permissions and more volume, then a tool call times out mid transaction and the retry writes the same record twice. The fix is not a better model. It is idempotency keys on write actions, a retry ceiling and an alert when a run exceeds its expected step count.

What control checklist can a small business run without a security team?

A small business can run a credible checklist by keeping it to what one operations lead can verify in a day. It is not a substitute for an audit. It is the minimum evidence set that lets you answer a regulator, an insurer or a nervous client without guessing. Start with an inventory: every agent in production, what it can reach, who owns it and what triggers it. Most businesses cannot produce that list, which is itself the finding. Then confirm each agent runs on its own credential with the narrowest scope that does the job, every irreversible action passes an approval gate with a recorded approver identity, and step logs are retained somewhere the agent cannot edit, because an agent with write access to its own audit log is not audited. Finish by running your adversarial set and recording the date, the version tested and the result.

What will auditors and regulators ask for?

Auditors and regulators will ask for three things: a record of what the agent did, evidence that a human was accountable, and proof the data stayed where you said it stayed. Australian obligations make that concrete. APRA regulated entities must show operational risk and third party arrangements are controlled. The Privacy Act requires personal information is handled only for the purpose it was collected, and records obligations require business records are retained and retrievable.

The question that catches people is not whether the agent was accurate. It is who was accountable. If an agent sends a wrong quote, a wrong payment or a wrong eligibility decision, the regulator wants to know which human signed off on the process that allowed it. The approval gate has to record an identity, not just a timestamp, and the log has to survive a challenge. If the agent runs in a vendor cloud and you cannot export the trace, you have a control you cannot evidence. That is also a commercial problem, because an Australian client in financial services, health or legal work will ask for the audit trail before they sign.

Why does running agents on-prem change the governance maths?

Running agents on-prem changes the governance maths because logs, prompts and tool results never leave the building. That removes a whole class of questions. You do not need a data processing agreement for inference, you do not need to prove where a trace is stored, and you do not need to explain which subprocessor saw a client document. The evidence sits on hardware you control, under your retention policy.

On-prem also changes the cost curve, which matters because spend is the other half of the governance question. An agent on a machine you own has a fixed monthly cost rather than a per token bill, so a runaway loop costs you time rather than money. Flowtivity runs dual DGX Spark machines with DeepSeek V4 Flash locally for exactly this reason: client data stays onshore, inference costs are predictable, and the audit trail is a file on a disk you own. The trade-off is real. You take on model updates, hardware lifecycle and the discipline of running a small inference stack, which is why the cost model deserves its own analysis first. For a business handling no sensitive data at low volume, a hosted model with good export controls is still the pragmatic choice. The point is to decide deliberately rather than by default.

What is the Flowtivity play?

The Flowtivity play is to make agent production boring. We build the control layer before the agent touches live systems, so the first conversation with an auditor or a client is about evidence rather than excuses. Most engagements start with a governance review of an agent that already runs, because that is where the risk usually sits.

Four offers cover the work. First, an agent governance audit: we inventory every agent in production, map its permissions and produce a gap list against the four controls. Second, a runtime control build: scoped credentials, approval gates and step logging wired into your existing agent, with the trace stored where you can export it. Third, an adversarial test pack: a repeatable set of prompt injection, tool poisoning and boundary tests you can run before every release, with a written record of each run. Fourth, an on-prem agent deployment: a local model stack on hardware you own, sized to your workload, with the audit trail and retention policy configured from day one.

Frequently asked questions

Do I need enterprise tooling to keep an AI agent under control?

No. Four controls cover most of the risk and none of them require an enterprise contract: scoped permissions, human approval gates, full step logging and adversarial checks. Enterprise platforms make those controls easier at scale, but a small team can implement the same four with its existing identity provider, a queue and a log store it controls.

What is the difference between agent monitoring and agent observability?

Monitoring tells you whether the agent is up and whether a run failed. Observability tells you why. It records every step of a run, including tool calls, arguments, results, timings and cost, so you can replay a session and attribute an outcome to a specific decision. Monitoring catches outages. Observability catches the decisions that produce bad outcomes while everything looks healthy.

How much does it cost to run AI agents in production?

More than the pilot suggests. A Splunk survey of 500 engineering leaders put average AI spend near US$85,500 a month, and Gartner found pilots run only 15 to 25% of the real production bill. Our arithmetic on that average is about US$1.03 million a year. Those are vendor-surveyed figures, so use them as a planning signal and measure your own token and tool spend per agent before you scale.

Can I keep agent logs in Australia?

Yes, and for regulated work you probably should. Running the agent and its trace store on infrastructure you control keeps prompts, tool results and audit records onshore, which simplifies Privacy Act and APRA conversations and removes the need to explain offshore subprocessors. Flowtivity runs local models on dual DGX Spark hardware for clients who need that property.

What should an approval gate cover?

Every irreversible action: moving money, deleting or overwriting records, emailing a customer, publishing content and changing production configuration. The gate should record who approved, what they saw and when, because accountability is the question auditors actually ask. Reversible read actions can usually run unattended.

Takeaways

• Production is a custody problem. If you cannot show what the agent saw, called, cost and who approved it, you are not in control.
• Four controls do most of the work: scoped permissions, approval gates, full step logging and adversarial checks before each release.
• Pilots hide the real bill. Vendor surveys put average AI spend near US$85,500 a month and pilots at 15 to 25% of production cost.
• On-prem agents keep the trace inside the building, the simplest way to answer APRA, Privacy Act and records obligations.