Small models matched a 70B behavioural benchmark. Then the tasks changed

Seventy billion was not the whole story

A large model can be a useful scientific instrument. It can also make the instrument look more expensive than it needs to be.

A paper published for the 2026 Conference on Language Modeling asks how small a model can get while still predicting choices in psychology experiments. The authors trained 14 base models, ranging from 135 million to 14 billion parameters, on Psych-101.

That dataset contains 10.7 million trial-level choices from more than 60,000 people across 160 experiments. The models see a participant's session as text and learn to predict the next choice.

On held-out participants doing familiar kinds of experiments, the answer was surprisingly small. With enough adapter capacity, models around 600 million to one billion parameters matched the team's reproduction of Centaur, a 70B-parameter baseline trained on the same data.

Fine-tuning did more work than scale

The researchers varied three things separately: the size of the base model, the capacity of the low-rank adapter used for fine-tuning and the amount of training data.

Within Psych-101, adapting a model to the behavioural data mattered roughly ten times more than making the base model larger. The models clustered in a narrow performance band once they had been fine-tuned.

The exact size needed to match the 70B reference changed with the adapter. At the lower adapter setting used by the original Centaur work, models between three and eight billion parameters reached the mark. With more adapter capacity, a 600M model could get there.

So the result is not that size never matters. It is that, on a well-covered set of experiments, a focused training signal can do much of the useful work.

New experiments brought scale back

The picture changed on Psych-201-RT, a set of 18 experiment types the fine-tuned models had not seen during training.

The tight group opened up. Larger models in the matched Qwen and Llama families kept much more of their original advantage when they had to transfer to a new task structure. Fine-tuning flattened the size gap on familiar experiments, but not on unfamiliar ones.

That distinction matters for how a research team chooses a model. A small system may be enough to estimate how predictable people are within a known paradigm. A broader instrument, expected to travel between quite different tasks, may still need more capacity.

It also prevents a cheap headline from swallowing the study. Small matched large, yes, but only inside the world the training data had already mapped well.

The models were not just replaying choice history

A second question hangs over behavioural prediction: does the model understand anything about the task, or has it found a shortcut in the sequence of earlier answers?

The authors progressively removed the instructions, the experimental stimuli, the outcome feedback and the structure around the choice history across 27 experiments. Masking the content of stimuli and feedback destroyed 75.7% of the information the models had learned and pushed average performance below chance.

They also shuffled trial order. Predictions stayed stable where each trial was independent and changed where earlier responses shaped what came next. The pattern suggests that the models used information in a way that reflected the experiment's structure.

That is evidence against one simple shortcut. It is not evidence that the models think like people or contain a readable theory of the mind.

What is confirmed, found and still open

Confirmed: the study appears as a COLM 2026 conference paper and was posted to arXiv on 5 August. Its training code, evaluation outputs and fine-tuned adapters are publicly available.

The research finding: small, cognitively fine-tuned models could match a reproduced 70B baseline on held-out participants from familiar experiments. On unseen experiment types, the benefit of scale returned. Prompt ablations also showed that stimulus and feedback content carried most of the learned signal.

The authors' interpretation: these systems can serve as noise-ceiling estimators, showing how much behaviour in a known experimental setting is predictable before a more interpretable theory has to explain it.

Still open: whether the same result holds with full fine-tuning, other architectures or datasets that follow the same people across different tasks. The study uses dense decoder-only transformers and participants are nested inside individual experiments.

Most importantly, prediction is not explanation. These models reproduce laboratory choices without specifying the mental mechanism behind them. A smaller proxy can still be useful. It should not be mistaken for a smaller theory of a person.