Who can stop your AI when something goes wrong?

The question to ask before handing over more work

Imagine an AI assistant helping a service business update customer records and prepare follow-up messages. The work is routine, the team is busy and the assistant is saving time.

Then an employee notices changes to records outside the assistant’s assigned work. They close the chat. More changes appear.

In this hypothetical scenario, the assistant has already handed a batch of requests to the customer-management system. Closing the conversation ends the visible interaction, but the other system continues processing accepted work.

The business now needs answers: who can intervene, which customers are affected, and how can staff continue serving them while the problem is investigated?

Our view is that the ability to contain an AI system belongs in the original adoption decision. It should be demonstrated before the business depends on the system to perform consequential work.

Stopping the conversation and stopping the consequence are different events

An AI agent is an application that can use tools to perform tasks. Its work may involve several connected services. One service starts the task, another changes records and another schedules a message for later delivery.

A stop control may prevent the next task while leaving previously accepted work active elsewhere. The exact behaviour depends on the product and configuration. A leader should ask the provider to explain the limits in terms of the business activity being stopped.

For example, “we can disable the assistant” is an incomplete answer if the concern is whether customer messages will still be sent. The more useful answer identifies who can suspend sending, what happens to pending messages and how completion or cancellation is confirmed.

NIST’s Generative AI Profile calls for defined response and recovery procedures and clear responsibilities for deactivation.^[1] Those responsibilities become meaningful when they connect to the actual services your organization uses.

Plan for a controlled interruption

There is a real trade-off. Suspending an assistant may slow service, create a backlog or require staff to take over. Leaving an unsafe capability active may create greater disruption. Decide in advance which circumstances justify immediate intervention and what essential work must continue.

A small service business might permit its operations manager to suspend automated record changes while staff handle urgent requests manually. General information lookups may continue if they can be separated safely. The technical provider supports the suspension and confirms what happened to outstanding jobs.

This is more actionable than a general promise of human oversight. The named responder needs working access, a backup and an escalation route when the supplier controls a critical function.

The supplier’s role deserves attention before contracting. If only its support team can stop a workflow, establish the support coverage, expected response arrangements and evidence it can provide. An escalation contact alone does not demonstrate an effective stopping mechanism.

Three preparations that make intervention credible

1. Give someone authority to act. Identify the business triggers for suspension, the person empowered to act and their backup. Confirm that the relevant controls are accessible before an incident.
2. Ask for a demonstration of what stops. In a controlled exercise, have the team show how it stops new work, active activity and requests already accepted by connected systems. Use harmless test records and confirm the effects where the work actually happens.
3. Agree how service resumes. Define who reconciles changed records, who decides whether corrections are needed and who approves the restart. Keep unresolved capabilities restricted and establish how the initial resumed work will be monitored.

For the service-business example, the exercise may show that pausing the assistant prevents new requests while five accepted jobs remain pending. That finding gives the team a specific gap to address before expanding the workflow.

NCSC guidance similarly links AI deployment to incident preparation, trained responders and useful audit information.^[2] The point of the exercise is to establish what your organization can actually do.

Treat restart as a business decision

A restart should have a defined scope. The owner may restore information lookups while keeping updates suspended. Previously accepted jobs should be reviewed before replay, and changes should be reconciled carefully so corrections do not overwrite legitimate work.

Some consequences cannot simply be reversed. An external message already delivered or confidential information already disclosed needs a response beyond turning the system off and on. Bring those effects into the existing incident process.

Start with one workflow that can change a record, send information or make a commitment. Ask the owner and provider to walk through the intervention and recovery together. Record the unresolved dependencies and the decisions they require.

If that workflow went wrong today, could your team stop the business consequence and explain what happened next?