Artificial intelligence systems are beginning to do more than recommend an action. They can approve purchases, close support cases, modify workflows and interact with other software without waiting for a person to authorize every step.
Gartner predicts that agentic AI will make at least 15% of day-to-day work decisions autonomously by 2028, up from effectively none in 2024.
That shift could make businesses faster and more efficient. It also creates a difficult question when something goes wrong: Who authorized the decision, what information did the system use and was the action permitted?
Khushan Adatiya, a senior software engineer at Google, believes companies need to answer those questions before autonomous systems become deeply embedded in daily operations.
Adatiya has worked on security, compliance and governance systems for large cloud platforms. Those systems are designed to define what software may do and create records that can later show whether controls worked as intended.
“For 20 years, the hard question in enterprise software was not whether it ran,” Adatiya said. “It was whether you could prove what it did when someone asked.”
Autonomous software changes the risk
Traditional enterprise AI generally provides information to a human decision-maker. It may flag suspicious activity, rank potential customers or recommend a price, but a person remains responsible for the final action.
AI agents can remove that person from the immediate process. An agent may review information, select a tool and complete several connected actions before anyone examines the result.
That autonomy is the feature companies want, but it also weakens accountability systems built around human approval.
An agent may choose different approaches when given similar tasks. It can also carry information from earlier interactions or coordinate with other agents. This makes its behavior harder to predict than that of conventional software following a fixed set of instructions.
Companies therefore need controls that govern the boundaries of an agent’s authority, not merely a list of expected actions.
Audit trails need to explain decisions
Keeping logs is not enough if those records cannot explain what happened.
A 2026 survey of 525 U.S. professionals found that 74% believed their organizations could pass an AI compliance audit. Only 27% described their AI governance programs as fully mature.
The survey also found that 46% of the organizations already had AI agents in production, while approaches to human oversight varied.
For an audit trail to be useful, it should show what information an agent received, which options it considered, what action it selected and whether that action stayed within company policy.
“A log nobody can reconstruct into a story is decoration,” Adatiya said. “The moment that matters is six months later, when an auditor or regulator asks why the system did a specific thing on a specific day.”
The record also needs to remain trustworthy. If logs can be changed, are missing context or cannot connect multiple actions, they may not provide meaningful accountability.
Some decisions still need people
One practical approach is to separate routine actions from decisions that carry greater risk.
An agent might be allowed to sort information, prepare a report or complete a low-value transaction independently. Actions involving sensitive personal data, large financial commitments or legal consequences could require human approval.
Those limits should be established before deployment. Companies should also define what happens when an agent encounters an unusual situation or attempts to operate outside its assigned permissions.
“People want a single product that makes agents safe, and there was never going to be one,” Adatiya said. “You decide in advance which calls an agent can make alone, you force the risky ones to stop for a human, and you keep a record clean enough to reconstruct any decision later.”
Permissions should also be limited to what each task requires. An agent that needs access to one database or payment function should not automatically receive broad access across an organization.
Compliance must be built into the system
Autonomous systems do not receive an exemption from rules governing privacy, data residency or automated decision-making.
An agent can cross a data boundary or complete a restricted action in seconds. That makes it difficult to correct compliance problems after deployment.
Adatiya argues that requirements such as regional data storage, approval thresholds and access restrictions should be built into the system’s architecture. He has also examined how large-scale systems maintain their guarantees under real production conditions in research available through IEEE Xplore.
The same principle applies to AI agents. A policy stating that sensitive information must remain in a particular region is only useful if the system technically prevents the agent from moving it elsewhere.
“Compliance is easy to describe and expensive to prove,” Adatiya said. “The work is building the system so those things are structurally true and you can show it.”
Building guardrails before agents multiply
Companies adopting autonomous AI can begin with a few basic controls. They can narrowly limit permissions, require approval for consequential actions, continuously monitor agent behavior and preserve records that explain each decision.
They also need a clear chain of responsibility. Someone must remain accountable for choosing the system, defining its authority and responding when it behaves unexpectedly.
These measures may slow an initial deployment, but adding them after thousands of agents are operating across an organization will be much harder.
“We are handing software real authority and assuming it will stay in bounds,” Adatiya said. “Assuming is not governing.”
©2026 Cox Media Group
