How to measure energy per inference

Define the unit first. Energy per request, per thousand tokens, per generated image, or per processed frame — each is valid, but comparisons are only meaningful within one definition.

Measure at the socket over a fixed window with a known request count, and subtract measured idle energy for the same window to obtain marginal energy. Report both figures: total and marginal.

Hold model, quantisation, batch size, context length and thermal state constant, and repeat runs. Report the median and the spread, not a single best case.

Publish your method alongside your numbers. Unmethodised efficiency claims are not comparable and should be treated with caution.