The Most Important Button in an AI Agent Is the One the Model Cannot Ignore

Imagine an AI agent halfway through organizing a company offsite.

It has opened the calendar, compared flights, emailed two venues and started a hotel booking. Then you notice the city is wrong.

You press Stop.

What exactly have you stopped?

The next sentence in the chat? The browser automation? A payment already submitted but not confirmed? An email sitting in a send queue? The agent’s access to your calendar? The hotel cancellation it may decide to make after realizing its mistake?

If the interface cannot answer those questions immediately, the Stop button is theatre.

That is becoming a real product problem as personal agents move from generating text to taking actions. Business Insider reported this month on early users deleting or restricting personal agents after privacy scares and costly mistakes. The Wall Street Journal documented cases involving a bank statement appearing in the wrong work conversation, mistaken reservations and unintended purchases. These are anecdotes, not a failure-rate study. But they expose the type of failure that ordinary chat UI was never designed to govern.

The most important control in an AI agent may therefore be the one the model cannot ignore.

Stopping the Chat Is Not Stopping the Agent

Chat interfaces have trained us to think of generation as a stream. Press Stop and the words stop appearing. The contract is obvious because the only thing happening is output on the screen.

Agents break that mental model.

A single instruction can expand into a chain of tool calls across systems the user cannot see at once. The agent might search the web, read a file, draft a message, reserve inventory, fill a form and trigger a payment. Some steps are local. Others leave the device. Some are reversible. Others become somebody else’s problem the moment they are submitted.

Stopping the visible conversation may therefore have no relationship to stopping the underlying work.

This gap is dangerous precisely because the interface looks familiar. A red square beside a streaming response carries years of learned meaning. Users reasonably assume it terminates the thing they asked to stop. If the product merely interrupts model generation while a queued browser action continues, it has borrowed the semantics of control without delivering the control.

The design requirement is not “add a stop icon.” It is define which system owns termination.

Pause, Cancel, Revoke, Undo

Agent interfaces need a vocabulary more precise than Stop.

Pause means do not begin another step yet. The current state remains intact and the task can continue.

Cancel means abandon the outstanding task and prevent queued actions from executing.

Revoke means remove the agent’s authority to use a tool, account or resource, regardless of whether the task is still open.

Undo means reverse an action that has already happened, if the external system permits reversal.

These controls overlap, but they are not interchangeable. Revoking calendar access does not unsend an email. Cancelling a purchase workflow may not reverse a charge that already cleared. Pausing a travel task should not silently leave a hotel room on hold forever.

A trustworthy interface has to show the user which of these states applies. Otherwise “Stop” becomes a comforting word placed over several incompatible realities.

The Control Plane Cannot Be the Model

Satya Nadella recently made a security argument that has a direct UX consequence. As reported by The Verge, Microsoft’s CEO argued that organizations should operate from the assumption that models can be compromised and rely on controls around them.

That is the right mental model for product design too.

A language model can recommend that an action should stop. It should not be the final authority on whether its own credentials remain active. The component deciding whether the agent may send the email, execute the payment or read the folder has to live somewhere more deterministic.

Microsoft’s Windows agent architecture offers a useful example. Its agentic security design gives agents distinct accounts, limited privileges and contained workspaces. The user can authorize, monitor and take over agent activity. The important part is architectural: the agent does not simply run as the human user with an unlimited version of their permissions.

This separation gives an interface something enforceable to control. Revocation can mean disabling a permission. Takeover can mean switching authority from the agent account to the person. An audit trail can distinguish what the agent changed from what the user changed.

Good agent UX will increasingly depend on these underlying primitives. Without them, the most elegant control surface is only a request.

Permission Should Expire With the Task

Today’s permission patterns are already strained by agents.

An app asks for access to email, files or a calendar, and the user makes a one-time decision. That pattern was tolerable when the app’s behavior remained relatively stable. An agent can use the same permission to perform an open-ended range of actions across changing contexts.

“Full email access” can mean reading one message to find a booking reference. It can also mean searching years of correspondence, downloading attachments, drafting replies and sending something to a recipient the user did not anticipate.

The interface should therefore authorize the task, not just the data source.

A useful permission could say: read these three messages, extract flight details, do not send anything, expire access when the itinerary is created. For a purchase: compare products freely, ask before creating an account, never exceed ₹20,000, require confirmation before payment.

That is more demanding than a single OAuth screen. It is also closer to the person’s actual intent.

DW’s earlier story about a local agent asking before it goes to the cloud pointed toward the same design principle. Trust improves when the boundary appears at the moment context changes, not months earlier in a settings page.

Receipts Matter More Than Reassurance

Agent products are remarkably good at narrating confidence.

“Done.” “I’ve taken care of that.” “Everything is booked.”

Those phrases are useful only when the interface can prove what they mean.

A transactional agent needs receipts, not merely summaries. If it sent eight messages, show the recipients. If it booked a room, show the confirmation number and cancellation terms. If it changed a file, show the version. If a task stopped midway, distinguish the actions that were prevented from the actions that had already crossed the boundary into another system.

This is especially important because the model’s own summary can be wrong. The evidence should come from tool state and external acknowledgements, not from the same probabilistic system describing what it believes happened.

That design principle also improves recovery. A user cannot reverse an action they cannot locate. A support team cannot investigate a mistake if every agent action collapses into one conversational paragraph.

DW’s analysis of agents bypassing visible interfaces raised a related problem: machine action can remove the cues people normally use to understand state. Agent receipts put some of that state back.

The Failure State Is the Product

The industry is understandably competing on how little supervision agents need.

A better model completes more steps without asking. It uses more tools. It keeps working while the user does something else. Every interruption can look like a failure of autonomy.

But high-stakes workflows invert the value of interruption.

When an agent is about to send an external message, transfer money, delete data or publish something publicly, a well-timed confirmation can be evidence of product maturity rather than model weakness. The goal is not maximum autonomy. It is maximum appropriate autonomy.

That changes how teams should test these products. Completion rate is not enough. Test what happens when the user changes their mind halfway through. Press Stop while a tool call is pending. Revoke a permission during execution. Disconnect the network after a purchase request. Give the agent an ambiguous recipient. Ask it to recover from a partially completed task. Then measure whether the user understands what actually happened.

A perfect happy path tells you very little about whether an agent deserves authority.

Autonomy Needs a Reverse Gear

The word “agent” implies initiative. Product teams have spent enormous effort improving that side of the experience: planning, memory, tool use, persistence and delegation.

The reverse direction is much less glamorous.

How does authority shrink? How does a task stop? How does the product hand control back? How does the agent explain what cannot be undone? How does a person repair the effects of a bad action without becoming an incident-response specialist?

These questions will determine whether agents become ordinary software or remain impressive demos that people use only for low-stakes tasks.

Microsoft’s approach of separate agent identity and contained workspaces is one clue. The wider security industry’s move toward zero-trust assumptions is another. The product-design implication is straightforward: an agent should never be the only system standing between its own decision and the user’s ability to contain it.

This also connects to DW’s coverage of enterprise concern around AI browsers. Broad capability without trustworthy containment makes organizations rationally cautious, even when the productivity upside is real.

The Button the Model Cannot Ignore

We are likely to keep calling it Stop because Stop is a word people understand.

But behind that word should be a much richer contract: halt future steps, freeze queued actions, revoke the relevant authority, show what already happened and offer reversal where reversal exists.

The visual control can remain simple only if the system beneath it is sophisticated.

That may be the defining UX challenge of agentic software. The industry’s ambition is to make machines capable of acting for us. The user’s requirement is smaller and more fundamental: when I say stop, prove that I am still the one in charge.