Self-improving agents can reward the mistakes they remember

A mistake can become a lesson too

An AI agent can improve from experience without retraining its underlying model. It keeps a memory instead: past tasks, past answers and a score saying which episodes were useful.

That sounds tidy until the agent grades its own work incorrectly.

A new preprint calls the resulting failure the Echo Gap. In the memory banks studied by the authors, some wrong episodes received unusually generous scores. The agent then brought those episodes back for similar tasks and trusted the same bad lesson again.

This is more than ordinary model noise. A weak answer does not disappear after one turn. It is written into persistent memory, ranked as useful and allowed to influence later behaviour.

A stronger judge did not automatically break the echo

A natural fix is to ask a stronger model to re-grade the memory. The study found that this was not enough in the tested banks.

The newer judges were generally better aligned with the true answer. But their remaining errors still tended to line up with the bias in the original self-grade. They were better judges overall, yet not independent enough where correction mattered.

The authors formalise that distinction as the Error-Independence Assumption. In their model, a corrective signal needs two properties: it should track whether the episode was actually useful, and its errors should not repeat the original grader's blind spots.

That is a useful design rule even without the name. If an agent, its memory scorer and its verifier all fail in the same way, adding another score may just make the system more confident.

The correction looked at behaviour, not the hidden answer

The researchers tested a targeted method called LUCID on BIRD, a benchmark where an agent turns ordinary questions into database queries.

LUCID did not see the reference SQL or the correct database result while the memory was being written. It checked signals available from the agent's own behaviour: whether a query failed or timed out, whether repeated runs disagreed, whether a result was empty or all null, and whether the query appeared to copy an entity from an unrelated memory.

Those checks had pooled precision of 0.90 on 649 flagged memories in the authors' analysis. Recall was low, so the detector did not try to catch every mistake. It tried to demote a smaller set it could identify with reasonable confidence.

The method changed the stored reward, not the text of the memory. That kept the agent architecture and model-call budget aligned with the self-graded baseline.

A small gain, measured over a full task stream

Across two task orders on the full 1,534-question BIRD development set, the memory-free agent reached 52.4% execution accuracy. The Memento-style self-graded memory agent averaged 54.0%. LUCID reached 56.9%.

The difference from ordinary self-graded memory was 1.9 percentage points in one seed and 3.9 points in the other. The reported paired confidence intervals excluded zero in both runs.

The paper also found that agent error rose as the retrieved set contained more wrong but trusted memories. With a clean retrieved set, the error rate was 0.32. With a fully corrupted set, it was 0.61 in the reported analysis.

Those numbers support the mechanism inside this experiment. They do not establish the size of the problem in coding agents, research agents or systems with a different memory policy.

What is confirmed, found and still open

Confirmed: the preprint, code, per-episode traces and run configurations are public. The end-to-end experiment used the full BIRD development set, two seeds and fixed model versions. Each arm saw the same task ordering within a seed.

The research finding: in the studied memory loops, self-grading overvalued some wrong episodes, correlated re-graders did not reliably repair them, and an answer-free targeted correction improved mean execution accuracy from 54.0% to 56.9%.

Still open: whether the Echo Gap appears at the same rate across other agent architectures, tasks, memory stores and model families. Two seeds are helpful, but still a small sample. BIRD also provides executable feedback that many real-world tasks do not.

The work is a preprint and has not been peer-reviewed. Model Current reviewed the paper and released artefacts, but did not rerun roughly 28,000 generation calls.

For teams building agents now, the practical question is simple: who grades the memory, and do they make the same mistakes as the agent that wrote it?