Why AI chip peak-performance numbers need a closer look

A recent reminder, not a new chip ranking

On October 6, MIT News described the Lincoln AI Computing Survey, an ongoing effort to map AI accelerator hardware. An accelerator is a processor or system designed to speed up demanding calculations.

The account points to a useful comparison tool, but not a fresh test of every chip available today. The cited paper was posted to arXiv on October 23, 2025. Its main performance chart describes capabilities as of summer 2025. That date matters when reading it in 2026.

Read the unit before comparing the number

The survey collects peak performance and power figures from public material, including company benchmarks. It distinguishes individual chips, cards and complete systems. A larger system can contain more hardware, so a bigger number is not automatically evidence of a better individual chip.

The chart also separates numerical precision: the format used to represent a number during calculation. A speed figure for one format should not be silently compared with a figure for another. They describe different conditions.

The paper discusses memory movement as well as arithmetic. For a model that spends time waiting for data, a headline count of calculations per second cannot by itself explain the speed of a finished response.

A workload benchmark asks a different question

MLCommons describes its MLPerf Inference Datacenter suite as testing how quickly a trained model processes inputs and produces results. The tests specify datasets, quality targets and request patterns, with response-time constraints and throughput metrics.

Its Closed division requires the reference model, supporting comparisons under shared conditions. Its Open division allows a changed or retrained model. Results from those divisions should not be treated as interchangeable.

Where power is reported, MLPerf measures the full system at the wall during the test. MLCommons says that measurement is valid for the accompanying benchmark; a component's power rating is not the same measurement.

Use specifications to narrow the question

A sensible reading starts with four questions: what hardware is counted, what numerical format is used, what model or task is tested, and what power measurement accompanies it?

A peak-specification survey can help map the options. A workload test can answer a narrower performance question. Neither automatically tells a team how its own application will behave. That final comparison needs the same task and acceptance standard, with unsuccessful runs counted rather than quietly discarded.