Tokens per watt: the new metric of the AI Factory era

Since it was first introduced in 2007, Power Usage Effectiveness (PUE) has become the global standard for calculating the energy efficiency of a data center and continues to provide invaluable insight for data center operators.

However, the landscape has changed significantly in the intervening years. The data center industry has been completely redefined by the rapid rise of AI and the infrastructure that supports it. Many of today’s large-scale AI data centers are increasingly viewed as AI Factories and token-production facilities. These facilities face severe power constraints and operate within hard limits that are determined by regional power grids and caps set by local utility operators. In this environment, the success of modern AI Factories is determined by how productively they use the energy at their disposal.

As we transition to the AI Factory era, the economics of AI has evolved along with it. In this new era, tokens per watt is emerging as a valuable metric for evaluating the productivity of AI infrastructure. Viewed alongside PUE, tokens per watt provides a more comprehensive view of data center performance.

Understanding AI tokens and tokens per watt

Tokens are the building blocks of AI and behind-the-scenes mechanism for breaking down AI learning and response tasks. Every answer generated, image created, document processed or conversation handled by an AI model is made up of tokens. As a rough rule of thumb, one token is equal to about 0.75 of a word so 100 tokens equate to roughly 75 words. For agentic AI, tokens serve as a practical unit for measuring computational work and AI output.

While PUE is the ratio of total energy consumed by a data center to the energy required to run the IT equipment within it, tokens per watt measures the volume of AI-generated output produced per unit of power consumed.

The metric directly connects energy consumption and energy efficiency to work done. This is an important calculation in a landscape where power is an increasingly scarce resource. For AI Factories, increasing the number of tokens produced per watt can potentially help improve infrastructure utilization and reduce cost per token.

Of course, tokens per watt isn’t meant to replace PUE. Instead, it is viewed as a companion piece, allowing data center operators to examine energy efficiency through a separate and relevant lens in the AI infrastructure landscape. While PUE is an efficiency metric for data center facilities, tokens per watt is a productivity metric for the compute layer of your data center. The two metrics complement one another and play a vital role in determining the value-generating capacity of a data center facility.

Why tokens per watt has become so relevant in the AI Factory era

As the industry plans and develops a new generation of gigawatt-scale AI Factories, power has become the primary constraint and the ultimate bottleneck for scaling AI. It is not enough to look solely at how much energy IT equipment consumes, but how much output it produces. In this context, tokens per watt has become a necessary way of thinking as data centers must consider the fundamental relationship between energy input and useful output.

The concept of tokens per watt allows data center operators to:

• Identify windows where outdoor ambient conditions allow the thermal infrastructure to run at lower temperatures and reveal opportunities that are otherwise masked by design assumptions
• Use those lower temperatures to let chips run in overdrive, produce more tokens per watt consumed and turn thermal headroom directly into compute output
• Drive infrastructure and thermal equipment selection to maximize token production year-round while still holding peak PUE targets so that efficiency gains scale with demand instead of trading off against it

Tokens per watt can also provide alignment between stakeholders who consider the components of a facility through various lenses. Engineers understand watts and power usage. They are hyper focused on driving energy efficiency, optimizing cooling technology and increasing uptime. AI teams understand computational output, tokens and model performance. Since leadership understands value and return on investment, tokens per watt can function as a shared intersection that sits at the intersection of all three.

Driving efficiency for AI Factories at gigawatt-scale

As data center operators contend with ever-tightening power constraints, diverse AI workloads, rapid deployment demands and complex data center cooling requirements, Johnson Controls has released a series of global reference design guides. These guides outline how to build high-performance, scalable facilities from a thermal management perspective.

The recent design guide for an air-cooled chiller plant outlines an approach that offers the following critical outcomes and energy efficiency wins:

• 50MW returned to the AI Factory through the implementation of bifurcated loops for air- and liquid- cooling systems
• 32% improvement in annual energy consumption through the intelligent utilization of redundant chillers
• 20MW peak power savings by quantifying and mitigating heat island effects of air-cooled chiller plants
• Zero water usage saves over 12 million gallons daily by eliminating the need for cooling towers to produce facility water
• 30% Coefficient of Performance (COP) improvement and 27% fewer chillers from raising the chilled water temperature to support warm-water Technology Cooling System (TCS) loops

Each design guide in the series maps the full thermal chain, from capturing heat at the chip level to managing the final atmospheric rejection, enabling flexible, scalable solutions that can achieve industry-leading PUE and WUE.

FAQs

What is tokens per watt?

Tokens per watt is an AI-specific metric that is used to measure the amount of useful AI output that a data center facility produces per unit of power consumed. For AI Factories, tokens per watt shifts the focus from energy consumption to energy productivity.

What is Power Usage Effectiveness (PUE)?

Power Usage Effectiveness (PUE) is the primary industry metrics used to determine the energy efficiency of a data center facility. PUE is the ratio of total energy consumed by a data center to the energy required to run the IT equipment within it. The closer the PUE number is to “1,” the more efficient the data center. Leading AI Factories target values as low as 1.2 or below.

What is Water Usage Effectiveness (WUE)?

Water Usage Effectiveness is used to determine how efficiently a data center uses water relative to the electrical energy consumed. It is calculated by comparing the total water consumed by a data center to the energy consumed by its IT equipment. The units of WUE are liters/kilowatt-hour (L/kWh). Data centers can potentially reach a WUE of “0.” This means that zero water is used in the cooling process, usually by relying on air cooling or closed-loop systems.