One batch, several possible bills
An AI service rarely gives one user an entire GPU and waits. It groups requests together so the hardware stays busy.
That makes the service more efficient. It also makes the energy bill harder to divide.
The GPU exposes power for the batch as a whole. A common shortcut is to assign that total in proportion to the tokens in each request. A long prompt gets a larger share; a short one gets less.
A new preprint argues that the shortcut is often wrong when very different requests run together. Tokens count text. They do not fully capture how prefill, decoding and neighbouring requests interact on the hardware.
The researchers replayed every useful combination
The team built JouleShare, an offline measurement harness around the vLLM serving engine.
For groups of eight requests, it replayed every non-empty subset while sampling GPU power every 100 milliseconds. That let the researchers ask how much energy each request added across the different combinations in which it could join a batch.
They used a Shapley value to turn those measured marginal contributions into a per-request share. The idea comes from cooperative game theory: shared cost is divided according to what each participant contributes across every possible order of joining the group.
It is not something a production service could calculate exactly for every live request. The number of combinations grows too quickly. Here it provides an offline reference for judging cheaper rules.
Token allocation missed the tested shares by a wide margin
Across 16 combinations of model and workload, token-proportional attribution had a mean normalised L1 error of 0.440 under static batching and 0.458 under continuous batching.
Put less mathematically, the authors calculate that roughly one quarter of each tested batch's energy was charged to the wrong requests. Every misplaced joule appears once as an overcharge and once as an undercharge in the raw difference, which is why the interpretation is half the summed deviation.
The gap was larger in heterogeneous long-context and synthetic groups than in more uniform chat groups. It also appeared on NVIDIA A800, A40 and H100 GPUs and in models from 1.5 billion to 14 billion parameters.
One extreme case involved a long-prefill request that token counting assigned 2,527.6 joules. Its measured Shapley contribution was 974.6 joules in static batching and 980.1 joules under continuous batching.
A small calibration model got closer
JouleShare uses the expensive replay data to train a lightweight estimator called JCalib. At serving time, the estimator needs only cheap features the system already has, including separate prompt and generated-token counts and their position within the group.
Keeping prefill and decode separate mattered. Collapsing them into one total made the estimator much worse on unseen models in the authors' tests.
JCalib reduced mean normalised L1 error to 0.116 under static batching and 0.177 under continuous batching. The reported prediction overhead was about 0.003 milliseconds per request.
A calibration trained offline also transferred to unseen models, workloads, GPU types and group sizes, though the error rose in several of those harder settings. This is a measurement proposal, not a production standard yet.
What is confirmed, found and still open
Confirmed: the preprint describes 40,800 measured subset generations for its exact eight-request study, plus 83,456 sampled coalition replays for groups of 16. The main runs cover four open model families and four workload types on vLLM.
The research finding: in these controlled batches, total-token allocation was a poor proxy for each request's measured marginal GPU energy. JCalib came closer to the Shapley reference at very low prediction overhead.
Still open: whether the method holds on live arrival traces, multi-GPU serving, models above 14B and full data-centre power. The study measures active GPU energy, not CPUs, networking, cooling or the rest of the system.
Exact Shapley values are also counterfactual by design. They depend on replaying combinations that production did not actually run. They are a defensible definition of fair contribution, not the only possible definition.
For sustainability reports and internal chargeback, that distinction matters. A precise token count can still produce a very imprecise energy bill.
Sources
- Luo et al. - Request-Level Energy Attribution for Batched LLM ServingPrimary preprint submitted 11 July 2026. Source for JouleShare, the replay protocol, all error figures, cross-hardware tests, JCalib results and disclosed limitations.
- Kwon et al. - Efficient Memory Management for Large Language Model Serving with PagedAttentionPrimary vLLM systems paper for the serving engine used in the JouleShare experiments.
- Chung et al. - The ML.ENERGY BenchmarkPrimary benchmark paper for token-normalised inference-energy accounting, the practical baseline audited by the new study.
