Rank agents on history,
not users.
First surfaced in Tandemly Briefing — June 8, 2026.
Testing a new LLM agent usually means deploying it and seeing what happens. A team at Emory and Shanghai Jiao Tong built a different path: train a diffusion world model on historical interaction logs, then simulate the environment for any candidate agent before it ever reaches a live system.
Live deployment is
the only option most teams have.
The field of offline RL has spent decades on off-policy evaluation. None of it worked for LLM agents, until now.
Most teams that evaluate LLM agents do it live. Deploy the candidate, run it on real users or real systems, measure outcomes. When the cost of a bad interaction is low, that works. When the agent has write access, talks to customers, or makes decisions with real consequences, it is not a viable process.
The field of offline reinforcement learning developed "off-policy evaluation" (OPE) specifically to address this: estimate how a new policy would perform using only pre-collected logs from an existing policy, no deployment required. This has been a standard tool in robotics and recommendation systems for years. The problem is that all of it was built for continuous action spaces, where what the agent "would have done" can be modeled as a smooth function of state. LLM agents don't work that way.
An LLM agent reads the environment's current message, then generates a reply as discrete text, token by token. The reply is fundamentally conditional on observing the state first. You cannot write down what the agent would have said without actually asking it.
Diffusion-based OPE methods handle this by jointly diffusing states and actions across a full trajectory: they pre-generate both what the environment says and what the agent would reply in one pass. That assumption collapses for LLM agents. An agent reply is the output of an autoregressive model that reads the environment's message first. You cannot jointly generate them without breaking the causal structure of language generation.
One transition.
One denoising step.
Instead of modeling a full trajectory jointly, ADWM models each environment transition as a separate, independent denoising problem.
ADWM (Autoregressive Diffusion World Model) is a framework for estimating a new LLM agent's performance from pre-collected interaction logs. The core architectural choice: rather than diffusing a whole trajectory in one pass, it trains a diffusion model on single-step transitions. Given the conversation history and the agent's most recent action, the model denoises toward a plausible next environment response. One step. One prediction. That's the full world model.
Evaluation works in an alternating loop. ADWM generates an initial environment prompt. The candidate LLM reads it and produces its action, exactly as it would in production. ADWM takes that action and runs one denoising step to produce the next environment response. The agent reads that response and replies again. The loop continues until the task terminates. Average outcome scores across many such simulated episodes produce the performance estimate for the candidate policy.
Standard autoregressive world models produce entire rollouts by conditioning each step on all prior steps. A small error on turn 3 produces a slightly implausible state on turn 4, which generates an increasingly off-distribution state on turn 5, and so on. By treating each transition as an independent denoising problem rather than a continuation of a global trajectory, ADWM contains errors within individual steps rather than allowing them to propagate across the full rollout.
Accurate estimates
across diverse multi-turn tasks.
ADWM produces reliable value estimates and evaluation rankings where prior diffusion-based OPE methods fail entirely for LLM agents.
The key contribution is not a marginal improvement on a shared leaderboard; it's a method that works for LLM agents where all prior approaches fail. Prior OPE methods produce unreliable estimates for discrete autoregressive agents because they violate the causal ordering that language generation requires. ADWM's step-independence restores that ordering. The result is accurate value estimates and correct candidate rankings across diverse multi-turn agent tasks.
If the candidate agent being evaluated behaves very differently from the agents that generated the historical logs, the world model will be asked to simulate environment responses it wasn't trained on. Estimates degrade under high distribution shift between the historical policy and the evaluation policy. This is a structural property of all off-policy evaluation, not a failure specific to ADWM. The paper recommends using OPE estimates as a filter, then validating the top candidates with a small live test before full deployment.
This synthesis is based on the abstract and search-accessible materials; the full paper contains detailed benchmark tables and ablations not reflected here. The quantitative results presented in the paper should be consulted before any implementation decision. arxiv.org/abs/2606.05558
What this changes
for agent evaluation.
The practical path ADWM opens up is candidate pre-screening before any live exposure. Getting there requires collecting the right data now, before you need it.
Where to go
from here.
If you want to apply or build on this work.