A good AI answer can still hide a business risk

The briefing is accurate. Was the process acceptable?

A professional-services firm asks an AI assistant to prepare a client briefing. The result is accurate, well written and ready for the account manager. On that basis, the demonstration appears successful.

Now consider a hypothetical detail that the finished document does not reveal: while preparing it, the assistant opened another client’s restricted file. It may also have saved a copy somewhere the original requester was not entitled to use.

The useful result remains useful. The unauthorized access remains a problem.

Our view is that organizations evaluating AI agents need to assess the authority exercised during the task as well as the quality of the final answer. This becomes particularly important when an assistant moves beyond drafting text and begins working across business systems.

What changes when AI can take actions

An agent can call functions that search documents, update records or send messages. These actions can have consequences before a person reviews the final response. The familiar habit of checking the finished document therefore covers only part of the work.

Google’s agent-evaluation guidance distinguishes assessment of tool-use sequences from assessment of final responses.^[1] For a business leader, the useful distinction is between an acceptable answer and an acceptable process.

You do not need access to a model’s private internal reasoning to ask what happened. The review concerns observable events: which records were read, which operations were attempted, what approvals applied and what receiving systems allowed or changed.

There may be several legitimate ways to complete a task. The aim is to establish whether the agent stayed within the approved boundaries, rather than insist that it always follows one fixed sequence.

Make the boundaries specific enough to assess

For the client-briefing example, the owner can define a simple permitted task: use the approved documents for the named client, prepare an internal draft and save it in the designated location. Sending it externally requires a separate approval.

That definition gives the team something concrete to test. Reading another client’s folder is outside scope. Saving an internal draft may be permitted. Sending it to an external recipient without approval is not.

The underlying account may still have access to a much larger collection. That is a gap to resolve with the technical owner, not evidence that every reachable record is appropriate for the task. OWASP’s excessive-agency guidance explains why functions and permissions should be limited to what is needed.^[2]

Business ownership matters here because a technical team cannot infer every acceptable commitment or audience from a product setting. The process owner defines the intended outcome and boundaries; the technical team demonstrates how the system enforces them.

Ask for evidence that answers a business question

1. Agree the permitted task before the demonstration. Identify the information, actions, recipients and approvals involved. Include one clearly prohibited action so the team can show how the boundary behaves.
2. Review what occurred alongside what was produced. Ask the reviewer to connect significant tool actions with the records, destinations and receiving-system results. The assistant’s own statement that it followed the rules is not sufficient evidence.
3. Turn uncertainty into a decision. If access was denied, record that result. If an operation completed outside scope, address the failure. If the relevant records are unavailable, state that the boundary has not been demonstrated and decide whether the use should remain restricted.

A lightweight review can start with one assistant, one workflow and a small set of synthetic records. Focus on consequences that matter: cross-client access, unapproved disclosure, unintended updates or commitments made without the required approval.

Use the findings to improve the next decision

The outcome should be a clear decision about the defined use. The client-briefing assistant might be approved for internal drafting after access is narrowed, while external sending remains unavailable. A later request to add sending would then trigger a separate review.

Keep the evidence proportionate and protect it. A business owner needs a readable finding and accountable next step; detailed logs can remain with the authorized technical reviewers. This avoids turning the review record into another uncontrolled copy of confidential material.

The same evidence is useful when choosing a supplier. Ask it to show how its controls apply to your proposed workflow and what it can demonstrate when an action is denied or completed. Product capability and the configuration you intend to use are connected, but distinct, questions.

When the next AI demonstration produces an impressive answer, will you also know whether the business boundaries held?