When the agent says done,
check the state.
On a structured airline benchmark, 78% of a budget agent's failures were tasks the agent reported as complete. The tools agreed. The database disagreed. A set of four deterministic read-only checks before each tool call recovered most of those failures without touching the model, the prompt, or the reasoning chain.
First surfaced in Tandemly Briefing — 2026-07-09.
The tool said yes.
The state said no.
When an LLM agent calls a tool, most systems treat tool acceptance as task completion. Those are two different things, and a large fraction of agents failures live in the gap.
Most agent architectures treat tool calls as the terminal action in a task. The agent reasons, selects an action, and the tool executes it. If the tool doesn't return an error, the task is done. The agent moves on, logs a success, and the next task begins.
That assumption breaks in domains where the policies governing an action live outside the tool's own enforcement logic. An airline booking API might accept a seat change without checking whether that change violates a fare rule stored in a separate policy database. A customer account tool might complete a refund request without verifying that the ticket meets eligibility conditions the tool wasn't designed to enforce. The API returns success. The downstream system is in a state that shouldn't exist.
This is the failure mode Reddy, Challaram, and Basu call a silent wrong-state write: the agent considers itself done, the tool confirms the action, and the resulting state violates a constraint none of the components in the loop were tracking. The failure is invisible without a dedicated check, and the agent's self-report is useless for detecting it.
If the dominant failure mode in a structured agent benchmark is silent wrong-state writes, not reasoning errors or tool rejections, can adding deterministic read-only verification before each tool call recover those failures without modifying the model or prompt?
Four gates,
zero model changes.
The researchers built a deterministic pre-execution layer that intercepts agent tool calls before any state change occurs and verifies them against the policies the tools themselves don't enforce.
The benchmark is tau-squared bench (tau-bench), a structured evaluation for LLM agents performing customer service tasks in an airline domain. Tasks involve booking changes, seat modifications, refunds, and related actions. Each task has a defined correct end state that can be verified programmatically, which makes it possible to detect silent failures that the agent itself doesn't surface.
The researchers first characterized the failure distribution. For a budget agent on tau-bench's airline domain, they found that 78% of all failures were silent wrong-state writes: the agent finished the task, no tool raised an error, and the resulting state violated a business policy. That number is high enough to reframe where the optimization effort belongs. If most of the failures aren't reasoning failures, then improving reasoning isn't the lever.
The four gates map to the four policy categories they found most frequently violated. Each gate is deterministic and read-only: it queries current system state via the tool's own read API, applies a rule, and either clears the pending action or blocks it with a specific reason code.
The gates are entirely outside the model's reasoning loop. The agent reasons the same way it always did. The gates intercept between the agent's tool selection and the actual tool execution. No prompt changes, no fine-tuning, no added reasoning steps. The verification is a separate layer that runs before state changes land.
The paper also ran negative controls: tasks where the tools themselves already enforced the same constraints the gates check. On those tasks, adding gates produced no improvement. This rules out the gates acting as a general-purpose accuracy boost and confirms they're filling a specific structural gap.
The agent thought
it was done.
The headline number isn't the performance gain. It's the 78%. Nearly four in five of a budget agent's failures were tasks the agent reported as complete, with no tool error to indicate otherwise.
On tasks where the tools already enforced the policies internally ("policy-permissive" tools in the paper's terminology), adding the four gates produced no meaningful improvement. This is a clean negative control: if gates were a general accuracy booster, they'd have helped on those tasks too. They didn't. The improvement on policy-permissive tasks was noise-level. This is evidence of a real causal structure, not an instrumentation artifact.
The statistical significance on gpt-4o-mini (P=0.0012) holds up under this reading. The gates are fixing something specific, not inflating the score globally.
tau-bench is a structured domain with measurable end states, which is what makes it possible to detect silent failures programmatically and define gates with precision. The paper doesn't claim this generalizes to open-ended tasks where "correct state" is ambiguous or where policies can't be expressed as deterministic rules. The gate definitions also require domain knowledge: someone has to identify which policies the tools aren't enforcing internally before the gates can be written.
Check what
the tool left behind.
The framing shift this paper asks for: stop treating tool acceptance as task completion and start treating it as one check in a chain. The other check is whether the resulting state is valid.
Where to go
from here.
If you want to apply this pattern or go deeper on the research.