An employee is reconciling a set of invoices when their AI assistant asks for permission to run a tool. The task is legitimate. The assistant has located a spreadsheet, found several mismatched entries and suggests sending the results to a service for checking.
The approval window says the assistant wants to “use an external tool.” There is an Approve button and a Cancel button.
What, exactly, is the employee being asked to approve?
The answer depends on information that may not fit neatly into a two-button prompt. Reading a spreadsheet is different from uploading it. Checking a supplier’s details is different from adding a supplier as a payment recipient. A person can understand the business task while still having no reliable way to judge the proposed technical action.
That is a problem with the decision presented to them, not evidence that they should have read the prompt more carefully.
Show the action, not just the tool name
A request to use a “file tool” could mean reading one document, changing permissions on a folder or deleting its contents. The label identifies an interface, not the consequence.
Before approval, the employee should be able to see the requested operation in ordinary language. Which file will be opened? What fields will be read or changed? Is the assistant copying information, creating a record or sending a message? Does the action affect one item or every item in a directory?
The same test applies to command-line actions. “Run a shell command” does not tell a reviewer enough. The exact command, working directory and relevant arguments matter. A command that displays a list of files is different from one that changes their ownership or deletes them.
If the interface cannot explain the operation, the person should not be expected to infer it from an unfamiliar tool name. The application can present a short plain-language summary, with the exact operation available for someone who needs to inspect it.
Check whose authority will be used
An AI assistant does not acquire the right to do something merely because its user has asked for a useful result.
Consider an assistant preparing a report. It may have access to a read-only project folder under the employee’s account, or it may be about to use a service credential with permission to modify records across the whole department. The visible task looks the same. The available authority does not.
An approval prompt should identify the account or service identity that will perform the action and, where it matters, the scope of that identity’s access. Is this a personal workspace? A shared company system? A production account? Does the permission last for a single operation or carry over to later requests?
This does not require every employee to understand an access-token format. It requires the system to distinguish between acting within the employee’s normal rights and using a broader privilege that deserves a different owner.
If the assistant needs more authority than the task normally grants, that is often a decision for the system or data owner, not the person who happened to receive the pop-up. This reflects the risk OWASP describes as excessive agency: AI systems having more permissions, functionality or autonomy than the task requires.
Make the destination visible
A second common ambiguity is where information will go.
Suppose the invoice assistant proposes sending a summary to an external checking service. The employee may recognise the vendor’s name, but the actual destination still matters. Is it the organisation’s approved service endpoint? A personal email address? A newly supplied web address? An external service that will retain the complete spreadsheet rather than only the mismatched invoice numbers?
For an outgoing transfer, the prompt should show the receiving service or address, the type of information being sent, and whether the destination is outside the organisation’s boundary. Where the destination is unclear, the right response is to verify it using an existing approved route.
The assistant’s confidence is not independent verification. Instructions found inside a document, website or message can influence what an assistant proposes. A destination written into one of those sources should not be treated as approved merely because the assistant presents it fluently.
Ask what will change, and whether it can be undone
Some actions are easy to inspect and reverse. Others create effects that cannot be recalled.
Reading an internal file may leave the source unchanged, but still expose sensitive information. Changing a draft can usually be undone. Sending an email, changing a supplier’s bank details, granting access to a folder or running a remote command can have consequences outside the assistant’s workspace.
A useful prompt tells the employee what will change if they approve. If the action can trigger a payment, publish information or modify a shared system, the interface should make that visible before the button is pressed.
An organisation also needs to decide which actions are not suitable for a simple conversational approval at all. A finance payment may require the existing finance approval workflow. Access to a sensitive repository may require a designated owner. The assistant can prepare the request without replacing the controls that already govern it.
Give people a practical way to decline
A warning without a workable alternative puts employees in an unfair position. Their immediate task is to finish the invoice reconciliation. If they cannot approve the tool call, they still need to know what happens next.
The interface should let them reject the request without losing the work already done. It should offer a way to narrow the action, such as using a redacted subset of records or keeping the processing inside an approved system. Where the decision belongs to another owner, the employee should know how to pass it on.
For the invoice example, a reasonable response might be:
“I can approve reading the authorised invoice file. I cannot approve uploading the complete file to an unverified external address. Use the approved finance tool or ask the finance system owner to confirm the destination.”
That is a useful security decision. It does not depend on the employee identifying a malicious intent or arguing with the assistant about whether it is trustworthy.
An approval button is not a security boundary by itself
The software hosting an assistant still has to enforce the decision. If a person denies a tool call but the assistant can perform the same action through another route using the same broad credential, the approval screen has not provided the protection it appeared to offer.
The Model Context Protocol tool specification recommends visible tool-use indicators and confirmation prompts, while leaving implementation details to the application. A confirmation prompt is part of the interaction design. Access control must still be enforced by the systems that invoke tools and own the affected resources.
The organisation should therefore be able to distinguish three things: the action that was requested, the decision that was recorded, and what the destination system actually did. A denied request is not proof that no side effect occurred unless the relevant execution and destination evidence support that conclusion.
This distinction matters when an operation times out or triggers background work. The assistant may display an error after the destination already accepted a change. The safer approach is to check the system of record, rather than assume the last chat message is the whole story.
Practise the decision before it becomes urgent
Teams can test this with two fictional approval screens.
One asks to read a named invoice file from an approved company drive, using the employee’s normal read-only access. It shows the exact file and does not send data outside the organisation.
The second asks to send a full copy of that file to a new external address using a broad service credential. Both claim to help with the same invoice task.
Ask staff what extra information they would need for the second request and who should own the decision. If the answer is not visible in the prompt or available through an ordinary workplace process, the exercise has found a design problem worth fixing.
The aim is not to train everyone to distrust useful assistants. It is to make the moment of approval match the consequence of the action. People should be able to see what will happen, under whose authority, where information will go and what will change. When they cannot, declining or escalating should be a normal way to keep the work safe.
Michael Kantor is President of HOL, an open-source organization working on standards and security for AI agents. His background is in software development, and his work focuses on how agents identify themselves, communicate and use external tools securely.
Professional profile: https://www.linkedin.com/in/kantorcodes.
HOL website: https://hol.org.
