What should AI agents be allowed to touch?

Last week, an AI coding agent wiped a production database in under ten seconds. Jeremy Crane, founder of an automotive SaaS platform, spent the weekend recovering it. Fortunately the data was restored, but his time was lost.

In the same week, Computer Weekly reported that the systems businesses use to control 'who can access what' are being rethought, because AI agents inherit a person's permissions yet can operate without the judgement or accountability that comes with them. Gartner has projected that the average Global Fortune 500 company will move from tens of agents today to more than 150,000 AI agents by 2028.

These are three isolated news items that, for me, point to the same underlying question. Not what these systems are capable of, but what they are permitted to do within a business once they are introduced.

Next week I am sitting down with an MD who wants to introduce agents into her company and is not sure where to begin. Her caution is justified, as is her desire to act. My view, going into that conversation, is that the starting point is not model selection or tooling. It is the scope of access the agent is given, and who in the business has the authority to define that scope.

In practice, that decision is often deferred. An agent is deployed against a narrow task, produces a useful output, and is then connected to additional systems so that it can operate with more context. Permissions expand incrementally. What begins as a contained function can quickly become part of the operational fabric of the company, without a corresponding decision about the limits of its behaviour.

For a typical SME, this is where risk can accumulate. The constraint on value is not the capability of the model. It is whether the business has defined, in concrete terms, the boundaries within which the agent will operate. In many cases, those boundaries are implicit. They reflect which credentials happened to be available when the agent was deployed, rather than an explicit view of what it should and should not be able to do.

When failure occurs, it tends to follow a consistent pattern. Read access is granted first to provide context. Write access is added later to improve efficiency. Each step appears reasonable in isolation. Taken together, they create a system that can act across multiple sources of truth without a clearly defined owner of the outcome.

Before an agent is connected to anything that affects customers, revenue, or core records, three points need to be established and recorded.

The first is the scope of interaction: which systems the agent can read from, and which it is permitted to change.

The second is accountability: the individual responsible for the consequences of those actions.

The third is reversibility: the mechanism by which the system can be returned to a known state, and whether that mechanism has been tested under realistic conditions.

If those points are not clear, then it can't truly be thought of as simply a failure of technology. It is that the business has not decided how it will use and govern it.

The incident involving Jeremy Crane will be read by many as a failure of a tool or a vendor. It is more usefully read as a scope decision that was never made. The same pattern sits underneath the concern about agents operating with inherited permissions. When these deployments fail, the temptation is to say that the system behaved unpredictably. However, it is better to ask 'who was asked to define the boundaries?'.