A test of reading the evidence
OpenAI introduced BioBasicsBench on November 6, developed with biotech company Ligo to test how agents interpret raw laboratory evidence. The benchmark contains 100 routine analysis tasks, including identifying errors and recognising when the data do not support a conclusion.
The task is to analyse supplied evidence, not to demonstrate that an agent can independently carry out a physical experiment. That difference matters when the phrase “AI scientist” is used to describe very different systems.
Partial credit is not a success rate
At the xhigh reasoning setting, OpenAI reports that GPT-6 Astra earned 66.7% of the available credit on average, compared with 43.8% for GPT-5.6 Sol. Answers are graded against criteria that can reward individual parts of an analysis.
The score is therefore not the percentage of experiments completed correctly, or even a count of wholly correct answers. These are OpenAI’s reported results, not an independent replication by Model Current. Selected examples also show assumptions overriding controls and observations being missed.
Why the reference check is a separate task
NIH’s standing guidance defines scientific rigor around well-controlled design, methods, analysis, interpretation and reporting. It also distinguishes a finding from one that other scientists can reproduce. That guidance provides context for reading a benchmark; it does not validate these particular scores.
In practice, spotting a pattern and deciding what it means are separate steps. A persuasive explanation can still rest on an unsuitable comparison. The relevant question is whether the conclusion follows from the evidence and its limits, not whether the answer sounds complete.
What remains to be demonstrated
The announcement is a useful account of a defined evaluation, not a certificate for unattended laboratory work. Independent reproduction, performance on a laboratory’s own data and the consequences of incorrect recommendations remain separate questions.
Readers should look for those answers before turning a benchmark result into an operational claim. No new laboratory experiments or model evaluations were performed for this article.
Sources
- OpenAI: Measuring how AI agents handle everyday wet-lab scienceNovember 6 primary BioBasicsBench announcement: 100 tasks, rubric-based partial credit, xhigh results and acknowledged errors. Developer-reported evaluation, not an independently replicated or autonomous wet-lab result. No experimental protocol reproduced.
- NIH: Enhancing Reproducibility through Rigor and TransparencyStanding primary guidance defining scientific rigor and reproducibility, last updated September 9, 2024. General context only; no NIH evaluation or endorsement of BioBasicsBench implied.
