Retrieval · Reinforcement Fine-Tuning · Mathematical Reasoning

When similar problems
teach the wrong lessons.

First surfaced in Tandemly Briefing — 2026-06-11.

A team at Rice University trained a retriever to find not the most similar math problems, but the most useful ones. Not "what looks like this problem?" but "what would teach someone how to solve this problem?" The distinction, it turns out, matters a lot.

Core concept
Reasoning-aware retrieval: selecting demonstrations by the quality of their solution strategy match, not by how similar the problem statement looks.
scroll to explore
01The problem

Semantic similarity is
not reasoning similarity.

Retrieval-augmented generation has a subtle flaw when used to supply training demonstrations for reasoning tasks. The way most retrieval systems choose examples is wrong for what these tasks actually need.

When you train a language model to reason better, one approach is to give it examples of other problems solved correctly. The model can observe the reasoning strategy in those examples and apply something similar to the target problem. This works well when the retrieved examples are genuinely analogous: same kind of reasoning, same structural move, similar mathematical machinery.

The catch is in how "similar" is defined. Standard retrieval systems, including the embedding-based ones that power most RAG pipelines, measure similarity in terms of topic and surface-level language. A problem about combinatorics retrieves other combinatorics problems. A problem involving quadratic equations retrieves other quadratic equations.

But the reasoning strategy that cracks a particular combinatorics problem might be a modular arithmetic trick that looks nothing like the retrieved examples. Meanwhile, a problem from a completely different domain might share the exact same structural approach. Surface similarity and reasoning similarity are not the same thing, and conflating them means you're feeding the model the wrong teachers.

The question this paper asks

What if the retriever was trained specifically to surface examples whose solution paths are useful for the target problem, rather than examples that look like the target problem? And what does that do for reinforcement fine-tuning on hard math?

02The experiment

Three stages,
one key shift.

The RA-RFT framework (Retrieval-Augmented Reinforcement Fine-Tuning) makes retrieval aware of reasoning utility. It does this across three stages, each building on the last.

1
Gold-relevance distillation
A capable judge model reviews pairs of math problems and assesses whether they share a transferable reasoning strategy, not just similar topic or wording. These assessments become explicit training labels: "this example is useful for the target problem; this one is not." This converts the fuzzy concept of analogous reasoning into concrete retrieval supervision.
2
Reasoning-aware retriever training
A dense bi-encoder retriever is fine-tuned on those utility-based labels via contrastive learning. After training, the retriever ranks candidate examples not by embedding distance but by predicted reasoning benefit. It learns to distinguish problems that share reasoning machinery from problems that merely share surface vocabulary.
3
RLVR with analogous demonstrations
The retrieved reasoning traces are injected into training prompts. The policy model (Qwen3-1.7B or Qwen3-4B) is then fine-tuned using GRPO, a reinforcement learning approach that rewards correct final answers on competition math. The model sees examples whose solution paths are structurally analogous to the target, giving it better raw material to learn from during the RL phase.

The retrieval corpus is OpenR1-Math-220K, derived from NuminaMath-1.5: AoPS Forum problems, AMC/AIME 1984 through 2023, MATH, and olympiad problems. Importantly, all of this source material predates the evaluation benchmarks, so there is no contamination of the kind that can inflate results in math benchmarks.

Standard RAG for training
Rank by semantic similarity. Retrieve the problems that look most like the target. Works well if looking-like and reasoning-like correlate. They often do not.
RA-RFT
Rank by reasoning utility. A judge model evaluates whether each candidate problem's solution strategy transfers. The retriever learns this judgment. The model trains on examples that are genuinely analogous.
What makes this feasible

Gold-relevance distillation is the enabling step. Labeling every pair of math problems by reasoning utility manually would be prohibitively expensive. Using a capable judge model to generate those labels at scale makes the approach practical. The distillation turns the retriever training problem into a standard contrastive learning task: bring reasoning-analogous pairs together in embedding space, push non-analogous pairs apart.

03Findings

Up to 7.1 points
on AIME 2025.

RA-RFT was evaluated on four competition-level math benchmarks against GRPO (the standard reinforcement fine-tuning baseline). Results were measured using avg@32: the average score over 32 sampled solutions at temperature 1.0.

AIME 2025 gain (Qwen3-1.7B)
+7.1
pp over GRPO baseline
AIME 2025 gain (Qwen3-4B)
+2.8
pp over GRPO baseline
Benchmarks evaluated
4
AIME 2024, AIME 2025, HMMT Feb 2025, BrUMO 2025
Headline results

On AIME 2025, RA-RFT outperformed GRPO by 7.1 percentage points on Qwen3-1.7B and by 2.8 points on Qwen3-4B. Averaged across all four competition benchmarks, the gains were 4.1 points (1.7B) and 2.6 points (4B). These are meaningful improvements on benchmarks where frontier models struggle to exceed 50% accuracy, and where any gain reflects genuine reasoning improvement rather than scale alone.

Orthogonality claim

The authors argue that reasoning-aware retrieval is orthogonal to reward design and curriculum: the retrieval improvement and the RL training objective are independent axes. This means RA-RFT gains can be stacked on top of other RLVR improvements without interference. If your team has been working on better reward shaping or smarter curricula for reasoning models, the retrieval component is an additive lever, not a replacement for what you've already built.

Scale behavior

Both the 1.7B and 4B variants of Qwen3 showed gains, which the authors take as evidence that the benefit comes from the retrieval structure rather than from properties specific to one model scale. The absolute gains are larger at 1.7B, which fits a consistent pattern in reasoning research: smaller models have more room to improve from better training signal.

Scope and limitations

The evaluation is limited to competition-level mathematical reasoning with verifiable rewards. Whether the approach generalizes to other domains where "reasoning utility" is harder to define and where no clear correctness signal exists remains an open question. The judge-model distillation step requires a capable model to generate reliable relevance labels. The approach is also a training-time technique: the retrieval investment occurs at training, not at inference, so it does not directly reduce inference costs.

04Practical takeaways

What this means
for builders.

RA-RFT targets a specific part of the training pipeline: which examples you retrieve to help a model learn. The implications are most direct for teams building and fine-tuning reasoning models, but the underlying principle about retrieval for training is broader.

1
For ML teams using RAG to supply training demonstrations
If you are selecting demonstration examples for reinforcement or supervised fine-tuning by semantic similarity, you are optimizing for the wrong objective on reasoning tasks. Semantic similarity retrieves topically related problems. Reasoning utility retrieves structurally instructive ones. These overlap but are not the same. Auditing whether your retriever actually surfaces examples whose solution paths are useful, rather than examples that look similar, is a concrete improvement axis you may not have examined.
2
For teams with existing RLVR pipelines
The retrieval component in RA-RFT is orthogonal to the rest of the training setup. If your team has invested in reward function design, curriculum scheduling, or sample weighting, the reasoning-aware retrieval improvement does not require you to revisit any of that. It is an independent axis. Adding it means running gold-relevance distillation on your corpus and retraining the retriever, not rebuilding the training loop.
3
For researchers working on hard reasoning benchmarks
The +7.1 point gain on AIME 2025 for a 1.7B model is a result on a benchmark where state-of-the-art systems still struggle significantly. The improvement comes entirely from what gets retrieved to support training, with the same base model, same reward function, same training budget. This suggests retrieval quality for training is a meaningful lever in a field that has mostly focused on model scale, architectural modifications, and reward design.
4
A note on scope
These results are on mathematical reasoning with verifiable correctness signals. Gold-relevance distillation works here because a judge model can reliably assess whether solution strategies transfer between math problems, and because AIME and olympiad problems have unambiguous correct answers. Applying this approach to domains where "reasoning utility" is ambiguous or where correctness is not verifiable requires more design work than is described in this paper.
05Further exploration

Where to go
from here.

Concrete next steps if you want to go deeper or try this approach on your own tasks.

1
Read the paper
Xiao, Ma, Chen, Chen, Atreya, Chen & Ordonez (2026). Learning to Reason by Analogy via Retrieval-Augmented Reinforcement Fine-Tuning. Rice University / Meta. arXiv:2606.13680.
2
Explore the retrieval corpus
The paper uses OpenR1-Math-220K, derived from NuminaMath-1.5. This corpus draws from AoPS Forum, AMC/AIME problems from 1984 through 2023, MATH, and olympiad problem sets. If you are applying RA-RFT to your own domain, the first task is identifying an equivalent corpus of solved problems with explicit solution traces.
3
Set up the gold-relevance distillation step
For each target training problem, sample a set of candidate problems from your corpus and ask a capable judge model to assess whether the candidate's solution strategy would help with the target. Gather enough of these labels (thousands of pairs) to train a contrastive retriever. The quality of this distillation step is the primary lever on RA-RFT's downstream performance.
4
Train the reasoning-aware retriever
A standard dense bi-encoder (an embedding model fine-tuned with contrastive loss on your labeled pairs) is sufficient. The innovation is in the training labels, not the retriever architecture. Evaluate your retriever on a held-out set of problem pairs before using it to select demonstrations for the full RLVR training run.
5
Compare against your existing retrieval baseline
Before a full training run, measure whether your reasoning-aware retriever actually retrieves more useful examples than a semantic-similarity baseline on a small validation set. If the retriever is not surfacing different examples, the distillation step did not work as intended and the downstream training will not benefit.