The answer can be wrong for two different reasons

When a language model gives a wrong fact, it is tempting to say it never learned the fact. A new Google Research paper asks a more awkward question: what if the fact is somewhere in the model's learned patterns, but the model cannot retrieve it when asked?

The authors call this the difference between encoding and recall. In their framing, encoding means the model can reproduce a fact in a context close to the one in which it might have learned it. Recall means it can reach the same fact through several direct or reversed questions. Recognition is easier again: choosing the right answer when it is already in front of the model.

Those distinctions do not make a model's internal state transparent. They do make a useful practical point. A confident wrong answer may come from a retrieval problem, not only from a missing-data problem.

What the paper tested

The team built WikiProfile from 2,150 facts drawn from Wikipedia pages. Each fact was paired with ten tasks, including prompts designed to test context-sensitive completion, direct and reverse questions, and multiple-choice recognition.

The paper reports testing 13 models with and without a thinking setting, sampling eight answers per model, fact and task. The resulting data set contains about 4.5 million responses. It is a substantial evaluation, though it remains a benchmark designed by the same group that advances the interpretation.

The research paper was first posted in February. Google Research published a fuller explanation on 12 August. Neither is a product promise or a claim that users should trust an answer because a model might secretly know more than it can say.

A useful result, with a narrow boundary

For the frontier models in this setup, the authors report that 95–98% of the tested facts were encoded, while 26–34% still failed on direct recall. With thinking enabled, the reported failure rate fell to 11–12%. They also found that rare facts and reverse questions were especially hard to retrieve.

That leads to a plausible engineering idea. If a system can tell when a question is likely to need more work, a reasoning step may recover part of what its basic response missed. The paper reports that thinking recovered roughly 40–65% of facts that were encoded but not directly recalled in the thinking-optimised models it tested.

But thinking is not a truth switch. It adds compute, can still produce an unsupported answer and cannot make an old training fact current. The paper itself notes that deciding when to invoke extra reasoning remains unresolved.

What this changes for everyday use

The result gives a better explanation for one familiar experience: a model can miss an ordinary fact when it is asked in one wording, then get it right after a little prompting or context. That is not proof of dependable knowledge. It is a reason to be cautious about treating one failed answer as a complete picture of the model's capability.

For important work, the dependable answer is still outside the model. Use a current source, ask for a citation, and check it. Retrieval from a maintained database has different strengths and failures from a model trying to retrieve a pattern from its parameters.

The research is useful because it separates those mechanisms. It should not be used as a reason to stop verifying facts.

What is confirmed, what the paper reports, and what is open

Confirmed: the authors' preprint, code-linked benchmark and Google Research explanation are public. WikiProfile contains 2,150 facts, and the paper describes 13 evaluated models and about 4.5 million sampled responses.

Measured and reported by the authors: in their benchmark, factual encoding was high for the named frontier models while direct recall remained materially lower. The specific percentages and the apparent benefit of thinking are results from that test design.

Open questions: whether the split holds on domains beyond Wikipedia, newer model versions, different prompt formats, independently built benchmarks and real user questions. The paper has not yet settled what a production system should do when it is unsure whether more reasoning will help.

Sources

  1. Calderon, Ben-David, Gekhman, Ofek and Yona — Empty Shelves or Lost Keys?Primary preprint. Source for the authors, benchmark design, model count, sampled-response total, findings and stated limitations.
  2. Google Research — Recall is the bottleneck for parametric factualityPrimary research-team explanation, published 12 August 2026. Source for the accessible methodology description and the reported recall-versus-encoding figures.