Analyst: AI Infrastructure Economics Hold – for Now

TLDR

• AI infrastructure economics remain viable despite rising costs.
• Surging token supply could eventually outpace demand.
• Enterprise ROI will determine whether the boom lasts.

Wall Street is understandably jittery about the enormous sums of capital being spent to build AI infrastructure – from chips, servers and storage all the way to data centers. Investors are concerned that all this spending will result in money-losing ventures.

Bloomberg Intelligence senior technology analyst Vasu Kasibhotla has crunched the numbers, and he believes the economics still hold. At current demand and pricing, the industry as a whole is absorbing the higher spending or passing along the cost to clients.

“Today, 100%,” Kasibhotla said in an interview with The AI Innovator. However, “the question is really not about today. It’s really about what happens two, three years from now.”

That’s when far more computing capacity comes online and the industry must find enough demand for the resulting flood of AI tokens.

For now, money keeps flowing into the AI infrastructure machine even as costs rise. Bloomberg Intelligence estimated about six months ago that a leading-edge, 1-gigawatt AI data center built around Nvidia GB200 NVL72 systems would cost roughly $37.3 billion. Its latest bottom-up estimate puts the cost in the mid- to high-$40 billion range as prices increase for memory, storage, networking, electrical equipment, cooling and construction.

The original $37.3 billion estimate included about $19.3 billion for server racks, $4 billion for networking and storage and approximately $14 billion for the physical infrastructure.

New generations of Nvidia hardware could push spending higher still. But there’s a silver lining.

Newer chips, although pricier, also produce more tokens so the cost per token could fall. The analyst cited Nvidia CEO Jensen Huang’s claim that its pricier Rubin chips could produce nearly 30 times as many tokens as the older Blackwell processors.

Many companies would be affected more by token costs, incurred from using AI models and tools, than the price of GPUs. This dynamic creates a potentially favorable equation as long as token output rises faster than chip costs.

A flood of tokens?

It also creates the biggest risk. If each new generation of AI infrastructure can produce vastly more tokens, the industry eventually has to find buyers for them.

Kasibhotla described that as the critical unresolved question. Increasing the supply of AI output doesn’t guarantee that customers will continue paying the same price for it. Greater capacity could drive token prices down, particularly if demand fails to keep pace.

“As you push all these tokens into the market, will there be sufficient demand for it?” Kasibhotla asked. “Because if that doesn’t exist, then a lot of these other things will not exist. They will not be supported.”

For now, infrastructure builders are operating on the assumption that demand will absorb the additional capacity. Kasibhotla said companies also face competitive pressure to invest because they fear rivals could gain an advantage if they have access to more capable AI.

Training has absorbed a large share of the computing infrastructure built during the past several years, he said. Competition among frontier model developers and other technology companies could keep training demand elevated even as inference – actually running models for customers – consumes increasing amounts of capacity.

Neoclouds find room to raise prices

An example of how AI’s economics function is to look at the neoclouds, companies whose core business is to provide AI computing capacity by renting out chips, servers and related equipment. They are a pure play on the AI infrastructure business.

“They are seeing inflation across the board,” Kasibhotla said.

But they are able to pass along costs to clients. Bloomberg Intelligence found that CoreWeave raised prices about 25% across its offerings in July while contribution margins on recent contracts rose by 5 to 10 percentage points despite higher component costs. Nebius, meanwhile, has been able to charge $40 million to $50 million per megawatt for agreements shorter than six months, roughly twice its rates on one- to three-year contracts.

The useful life of GPUs is also turning out to be important.

Early neocloud contracts had long terms, customer prepayments and committed usage to help them finance their GPU purchases. Those arrangements could lock operators into prevailing prices even as costs rise. Moreover, IREN’s five-year Microsoft agreement, for example, assumed the GPUs would have no residual value when the contract ended, according to Bloomberg Intelligence.

It turns out that older hardware remains in demand. CoreWeave has contracted 2020-era Nvidia A100 GPUs through 2029, according to the report. That means older GPUs can continue generating revenue even after newer chips become available. Higher residual values reduce risk for lenders and can allow neoclouds to sign shorter customer agreements, giving them more opportunities to raise prices as costs increase.

Chip suppliers can also help backstop some credit risk, allowing infrastructure companies to obtain financing without the suppliers directly funding an entire project, Kasibhotla said.

Enterprise spending remains uneven

Ultimately, however, the infrastructure chain depends on businesses finding productive uses for AI.

Data cited in Kasibhotla’s Bloomberg Intelligence report illustrates how far enterprise spending could potentially expand – and how uneven adoption remains.

Median AI spending was $11.95 per employee, compared with $650 among the top 10% of spenders and $7,400 among the top 1%, according to Ramp data cited by Bloomberg Intelligence. Paid AI penetration had reached 55.7% of U.S. businesses captured in the data in July.

Kasibhotla cautioned that the low median partly reflects the breadth of the sample, which includes smaller businesses. But enterprises also face a practical problem as they increase AI usage: controlling consumption.

Unlike traditional software licenses, AI services can be consumption-based, creating a new cost-management problem for companies.

“If you can’t measure, you can’t manage it,” Kasibhotla said.

At the same time, companies can’t simply stop employees from using AI if the reason for adopting it was to improve productivity. “So it’s not a trivial problem,” he said.

That makes enterprise ROI a potential constraint on the infrastructure boom. If businesses eventually decide the productivity or revenue generated by AI isn’t worth the cost, weaker demand could affect model developers, cloud providers and data-center operators.

One warning sign would be a rapid shift by customers toward cheaper open models and away from more expensive frontier models. Kasibhotla said such a shift could indicate that customers’ willingness to pay for higher-performing AI is weakening.

“That’s absolutely not good because then that means … the amount they are willing to pay for superior intelligence is reducing,” he said.

A gradual shift could be manageable, he said, but a sharper change could undermine investor confidence and ultimately restrict the financing supporting the infrastructure buildout.

“If the credit markets don’t provide the capital, then this whole thing will come down,” Kasibhotla said.