The AI Inference Crisis: How Reasoning Agents May Become an Energy Problem

AI has an energy conundrum: easy to misread.

Many of the debates about the environmental impact of AI have been about training giant models today for years. To train a frontier model, thousands of accelerators may be needed and an incredible amount of electricity, as well as advanced cooling systems may be required to process the information.

The economics of AI is changing though.

After a model has been trained, it must be used on the computer each time someone asks it a question.

This is referred to as inference.

As AI progresses from a chatbot to reasoning models and even independent AI agents, it’s possible that inference will be one of the most crucial computing challenges of the next decade.

It’s because modern AI is beginning to learn to allocate more compute resources to hard problems.

A reasoning model can produce thousands of intermediate tokens, consider alternatives, check back and attempt again rather than producing an answer right off the bat. An AI agent can make multiple calls to the models, utilize tools, read documents, write code, read the output and call models again.

The AI sector is thus presented with a basic dilemma:

There can be a significantly larger amount of computation involved in better reasoning.

But when the number of such interactions with an AI is in the billions per day, it can become an energy problem.


What Is AI Inference?

Inference is the act of making an output with a trained AI model.

The AI chatbot is not learning as you type to it. Rather, special hardware is used to calculate a response through an already trained model.

This does happen very fast but on a large scale it needs a tremendous infrastructure.

In a modern AI Data Center, thousands of GPUs or other AI accelerators can be packed into a single facility.A modern AI Data Center can feature thousands of GPUs or other AI accelerators in a single facility. They use electricity to carry out the mathematical calculations needed for AI models.

Electricity is not all used for the AI accelerator.

Also required for a production AI system is:

  • CPUs
  • memory
  • networking
  • storage
  • power conversion
  • cooling
  • backup systems
  • data-center infrastructure

According to the International Energy Agency, global electricity use for data centers would have reached around 415 TWh in 2024, which is about 1.5% of all global electricity use. Its base case assumes that by 2030, the electricity demand in data centers will increase to approximately 945 TWh.

One of the key drivers of that growth is AI.


Why Reasoning AI Changes the Equation

A conventional Chatbot that relies on artificial intelligence may be able to produce a response fairly directly.

A reasoning model acts differently.

It could take more time to calculate to arrive at the solution and then show the answer.

Take a look at a simple question:

What is the name of the capital of the France?

The model requires very limited computation.

Now consider:

“This 500-page financial report contains inconsistencies, the conclusions are conflicting with historical data, you’ll have to write some Python code to test the assumptions and you’ll need to come up with a recommendation.

That’s quite a different ballgame.

The model should be able to:

Read copious amounts of information.
2. Divide the problem into smaller tasks.
3. Generate intermediate reasoning.
4. Use external tools.
5. Code and run code.
6. Examine the output.
7. Detect mistakes.
8. Recalculate.
10. Give the final answer.

This is compute at test time: computation at test time to enhance the quality of an answer.

Test-time compute can have a radical impact on the energy landscape of AI.


This is the Token Multiplier Effect.

The number of tokens that are processed and generated is one of the most crucial variables in AI inference.

A basic answer to a chatbot could involve a comparatively small amount of generated tokens.

There can be thousands of reasoning models generated.

In 2026, Microsoft researchers projected an optimized frontierscale inference cost of approximately 0.31 Wh per query in their production-scale analysis in Joule. For queries with long reasoning or agentic workflows, however, energy use may be over an order of magnitude higher due to greater token generation and less energy efficiency at serve time.

Their modelled long-reasoning scenario had about 5,000 output tokens and resulted in an estimated energy consumption of about 13× that of a regular query.

This is a crucial point to keep in mind.

It’s not always the case that one AI question uses up a lot of electricity.

The problem is:

Millions or billions of questions × significantly more computation per question.

That times can get very big.


But AI Agents May Make it Even Worse.

Reasoning models are just part of the picture.

Perhaps the larger shift will be brought about by AI agents.

Generally, a chatbot goes through a fairly standard procedure:

User → AI → Answer

An agent may function more in this way:

User > Agent > Plan > Model > Tool > Model > Search > Model > Code > Model > Verification > Answer

Many model calls can then be triggered by a single user request.

For example, imagine asking an AI agent:

Find Best laptop below 1 lakhs, compare 20 models, check current price, analyse the reviews, calculate value for money, and recommend one.

The agent can do hundreds of operations.

Now imagine an autonomous software-development agent working for an hour.

It could:

  • inspect a repository,
  • identify bugs,
  • write code,
  • run tests,
  • inspect failures,
  • modify the code,
  • run tests again,
    Review and evaluate its own work,
  • and repeat.

There is more inference needed for each cycle.

A research paper shared at HPCA 2026 explored precisely this transition towards multi-step reasoning and found that agentic workflows introduce significant resource usage, latency, energy consumption and even data-center scale power usage concerns.


While that is indeed the case, it is far from an exhaustive list of different ways in which AI consumes electricity.That’s true, but it’s not the complete list of ways AI uses electricity.

To the extent that there is a nuance.

The existing claims on AI energy usage are derived from measurements taken on a system that is not representative of a production system.

In 2026, Microsoft published an analysis that identified that some popular AI-energy estimates greatly overestimate real-world inference energy by failing to include the efficiency of large-scale production systems (by up to 40×).

Google has also released measurements of their Gemini system. Under Google’s math, the median Gemini Apps text prompt used about 0.24Wh in May 2025.

The correct argument is not:

“Each AI inquiry uses a lot of electricity.”

It doesn’t.

The more interesting question is the evolving distribution of AI workloads.

Simple queries can be fairly efficient.

Computation can be orders of magnitude higher for long reasoning, video generation, complex coding and autonomous agents.

The International Energy Agency’s new analysis underscores this shift, revealing that newer AI applications with high energy requirements like reasoning and agentiveness can use much more energy to complete individual queries than text generation.


Why Billions of Queries Change Everything?

Assume that an AI system processes one billion queries every day.

If, at an illustrative average of 0.3 Wh per query:

1 billion × 0.3 Wh = 300,000 kWh

That’s:

300 MWh per day.

However, if a significant number of the requests start to grow more expensive to compute, the situation changes quickly.

According to the modeling by Microsoft, such a deployment serving one billion queries a day would consume around 0.7 GWh/day based on their baseline assumptions, with 10% of the requests being long queries pushing demand to around 1.7 GWh/day.

This is the crux of the issue.

The average query isn’t the only factor to consider for the energy footprint of AI.

It is based on the following:

  • how many users exist,
    What is the frequency of their usage of AI,
    What is the amount of computation needed to perform each task, and
  • the length of the reasoning process, and
    The number of agents involved,
  • how efficient the hardware is run, and
    The efficiency of a data center operation is indicated by the * and the efficiency of data center operation.

The data center is the new computer.The data center is the new computer.

AI inference is transforming the thinking around computing infrastructure.

The limiting resource for normal software may be CPU resources or storage.

When it comes to advanced AI, the limiting factor is becoming more and more a combination of:

GPU capacity, memory bandwidth, networking, electricity and cooling.

According to IEA, hyperscale data centers focused on AI applications can attain power ratings of up to 100 MW or more, with the biggest facilities in development being even larger.

This is a geographical problem as well.

Electricity demand is not uniform all over the earth.

A large AI data center can be in one geographical area, and put a strain on the local electrical grid.

Global electricity consumption from data centers is projected to more than double from approximately 485 TWh in 2025 to approximately 950 TWh in 2030, and electricity consumption in AI-focused data centers will be even greater.


The competition for “tokens per Watt” is underway.The race for “tokens per Watt” is on.

So, the question is not just to ask the AI hardware industry, but one also has to ask:

What is the speed of this GPUs?

A more important question is becoming:

How many useful AI tokens this system can produce for every unit of electricity?

This alters the design of the hardware.

A processor that generates a bit more output at a cost of hugely higher electricity usage may not be the optimal solution at data center scale.

Inference infrastructure is no longer just about metrics like throughput per GPU, but also throughput per megawatt, as in the case of NVIDIA.

This is especially true for reasoning models as generating thousands of tokens means that energy efficiency is always a concern.


Is there a solution to the problem that can be found in software?

There might be hardware enhancements required.

The goal of AI researchers is to try out methods to minimize the computing that happens.

An approach is adaptive reasoning.

An AI could determine the difficulty of the task and give each question a different amount of computation.

Simple question?

Use less compute.

Difficult mathematical problem?

Use more compute.

Complex agentic task?

Increase funding for this substantially.

The other one is model routing.

An AI platform could be designed to route simple tasks to smaller models, while reserving frontier-scale models for more challenging problems.

This will significantly lessen the unnecessary computation.


This research finds that multi-agent reasoning is also plagued by its own efficiency issues.

Interestingly, the more you add AI agents, the less efficient the system will be.

Various methods of multi-agent reasoning have been investigated and it has been shown that they make different trade-offs between accuracy and compute. In one study from 2026, ACL, some multi-agent approaches could yield more accurate results, but with significantly greater computational resources.

This raises another engineering query:

What is the cost of further computation for a small gain in accuracy?

If a second pass in the reasoning raises the accuracy by 2%, it might be worth while.

If it takes 10 times longer to compute for little gain it might not be.

Optimizing for intelligence per unit of compute will therefore be a key part of what happens in the future with AI infrastructure.


If efficiency is achieved, this could prevent an AI energy crisis.If efficiency can be attained, this could avert an AI energy crisis.

But there’s a significant reason why it’s not reasonable to expect that the energy use of AI will scale up endlessly.

Time and time again computing has been made more efficient.

The same can occur for AI inference.

The 2026 analysis by Microsoft suggests that improvements in model design, serving systems and hardware may lead to a factor of 8x to 20x reduction in inference energy under the scenarios studied.

The improvements may be made through:

  • more efficient AI accelerators, and
  • quantization,
  • model compression,
  • better scheduling,
  • improved memory systems,
  • speculative decoding,
  • mixture-of-experts architectures,
  • better batching,
  • more efficient cooling,
  • and smarter workload routing.

The issue is that the more efficient things become, the more people will use AI.

If AI does become more affordable to operate, individuals will use it more.

It is a good example of a technological rebound effect.


It is possible that the true problem with AI energy is the scale.The real issue with AI energy may be scale.

The biggest fallacy is that the issue with energy in the future is going to be a single, massive AI model.

It could be, rather, from billions of intelligent actions that are taking place all the time.

What if the world is a place where AI agents:

  • manage emails,
  • write software,
  • analyze financial data,
  • monitor cybersecurity,
  • generate videos,
  • operate robots,
  • conduct scientific research,
  • manage business operations,
    Communicate with other AI systems, continuously, and with the use of asterisks.

This number of inference operations could grow to be very large.

AI inference starts to look like a new type of digital electricity use at this point.

Data and GPUs are not the only commodities of the AI industry.Data and GPUs are not the only resources to the AI industry.

It could be the efficient calculation of energy.


In summary, the next generation of artificial intelligence is “Intelligence per Watt”.

Conclusion

The next generation of AI is “Intelligence per Watt”. AI’s evolution is in a new era.

The first era was mostly focused on training larger models.

The next age is about making models think longer, and act autonomously.

That shift poses a new infrastructure problem.

Reasoning requires computation.

Agents call the model, and add to the model calls.

More length of output, more tokens are generated.

The more the users the more is the total demand.

All of this eventually materializes in the physical world, in the form of GPUs, memory, networking, cooling systems and electricity!

However, it is not a foregone conclusion that AI will be an environmental disaster.

The technology is also getting a lot more efficient by the day as well.

The more useful question is, however, not:

Does Artificial Intelligence Use Electricity?

Of course it does.

The question is:

How much intelligence does it take to make from every watt of electricity?

This may turn out to be one of the most significant AI indicators.

The future winners for the companies developing the most powerful AI may not be the ones you’d expect.

The winning companies might be those that develop AI systems that can provide the most useful reasoning, while using as little computation or energy.

The AI race could thus become a race for the smaller models, or even move to a more fundamental level:

A race for “intelligence per watt.