Skip to contentSkip to site index

How A.I. Is Changing the Way the World Builds Computers

By Mike Isaac and Electricians are flocking to areas where the facilities are being erected. And in some places, locals are

Google’s A.I. chips on a circuit board. The company needs thousands of these chips to build its chatbots and other A.I. technologies.

Christie Hemm Klok for The New York Times

The bigger-is-better mantra was challenged in December when a tiny Chinese company, DeepSeek, said it had built one of the world’s learn skills by analyzing large amounts of data. Neural networks are the basis of chatbots and other leading A.I. technologies.

How A.I. Models Are Trained

By analyzing massive datasets, algorithms can learn to distinguish between images, in what's called machine learning. The example below demonstrates the training process of an A.I. model to identify an image of a flower based on existing flower images.

Sources: IBM and Cloudflare

The New York Times

In the past, computing largely relied on chips called central processing units, or CPUs. These could do many things, including the simple math that powers neural networks.

But GPUs can do this math faster — a lot faster. At any given moment, a traditional chip can do a single calculation. In that same moment, a GPU can do thousands. Computer scientists call this parallel processing. And it means neural networks can analyze more data.

“These are very different from chips used to just serve up a web page,” said Vipul Ved Prakash, the chief executive of Together AI, a tech consultancy. “They run millions of calculations as a way for machines to ‘think’ about a problem.”

So tech companies started using increasingly large numbers of GPUs to build increasingly powerful A.I. technologies.

Difference between CPU and GPU-powered computers

Sources: Nvidia, IBM and Cloudflare

The New York Times

Along the way, Nvidia rebuilt its GPUs specifically for A.I., packing more transistors into each chip to run even more calculations with each passing second. In 2013, incurred a $4.2 billion restructuring charge, partly to redesign many of its future data center projects for A.I. Its activity was emblematic of a change happening across the tech industry.

A.I. machines need more electricity. Much more.

New data centers packed with GPUs meant new electricity demands — so much so that the appetite for power would go through the roof.

In December 2023, Cirrascale leased a 139,000-square-foot traditional data center in Austin that drew on 5 megawatts of electricity, enough to power about 3,600 average American homes. Inside, computers were arranged in about 80 rows. Then the company ripped out the old computers to convert the facility for A.I.

The 5 megawatts that used to power a building full of CPUs is now enough to run just eight to 10 rows of computers packed with GPUs. Cirrascale can expand to about 50 megawatts of electricity from the grid, but even that would not fill the data center with GPUs.

And that is still on the small side. OpenAI aims to build about five data centers that

Cirrascale’s data center in Austin, Texas, draws on 5 megawatts of electricity, which can power eight to 10 rows of computers packed with GPUs.

Christie Hemm Klok for The New York Times

It’s not just that these data centers have more gear packed into a tighter space. The computer chips that A.I. revolves around need far more electricity than traditional chips. A typical CPU needs about 250 to 500 watts to run, while GPUs use up to 1,000 watts.

Building a data center is ultimately a negotiation with the local utility. How much power can it provide? At what cost? If it must expand the electrical grid with millions of dollars in new equipment, who pays for the upgrades?

Data centers consumed about 4.4 percent of total electricity in the United States in 2023, or more than twice as much power as the facilities used to mine cryptocurrencies. That could triple by 2028, according to a December report published by the Department of Energy.

Power consumption by A.I. data centers

The Energy Department estimates that A.I. servers in data centers could consume as much as 326 terawatt-hours by 2028, nearly eight times what they used in 2023.

Source: Lawrence Berkeley National Laboratory, Energy Department

The New York Times

“Time is the currency in the industry right now,” said Arman Shehabi, a researcher at the Lawrence Berkeley National Laboratory who led the report. There is a rush to keep building, he said, and “I don’t see this slowing down in the next few years.”

Data center operators are now having trouble finding electrical power in the United States. In areas like Northern Virginia — the world’s biggest hub of data centers because of its proximity to underwater cables that shuttle data to and from Europe — these companies have all but exhausted the available electricity.

Some A.I. giants are turning to nuclear power. Microsoft installing their own gas turbines at a new data center in Memphis.

“My conversations have gone from ‘Where can we get some state-of-the-art chips?’ to ‘Where can we get some electrical power?’” said David Katz, a partner with Radical Ventures, a venture capital firm that invests in A.I.

A.I. gets so hot, only water can cool it down.

These unusually dense A.I. systems have led to another change: a different way of cooling computers.

A.I. systems can get very hot. As air circulates from the front of a rack and crosses the chips crunching calculations, it heats up. At Cirrascale’s Austin data center, the temperature around one rack started at 71.2 degrees Fahrenheit on the front and ended up at 96.9 degrees on the back side.

If a rack isn’t properly cooled down, the machines — and potentially the whole data center — are at risk of catching fire.

Just outside Pryor, a farm-and-cattle town in the northeast corner of Oklahoma, Google is solving this problem on a massive scale.

Thirteen Google data centers rise up from the grassy flatlands. This campus holds tens of thousands of racks of machines and uses hundreds of megawatts of electricity streaming from metal-and-wire power stations installed between the concrete buildings. To keep the machines from overheating, Google pumps cold water through all 13 buildings.

In the past, Google’s water pipes ran through empty aisles beside the racks of computers. As the cold water moved through the pipes, it absorbed the heat from the surrounding air. But when the racks are packed with A.I. chips, the water isn’t close enough to absorb the extra heat.

Conventional Data Center

Source: SimScale

The New York Times

Google now runs its water pipes right up next to the chips. Only then can the water absorb the heat and keep the chips working.

A.I. Data Center

Source: SimScale

The New York Times

Pumping water through a data center filled with electrical equipment can be risky since water can leak from the pipes onto the computer hardware. So Google treats its water with chemicals that make it less likely to conduct electricity — and less likely to damage the chips.

Once the water absorbs the heat from all those chips, tech companies must also find ways of cooling the water back down.

In many cases, they do this using giant towers sitting on the roof of the data center. Some of the water evaporates from these towers, which cools the rest of it, much as people are cooled when they sweat and the sweat evaporates from their skin.

“That is what we call free cooling — the evaporation that happens naturally on a cool, dry morning,” said Joe Kava, Google’s vice president of data centers.