The tech industry is undergoing its most profound transformation since the dawn of the commercial internet in the 1990s. Just as companies once completely rebuilt their systems to accommodate the dot-com boom, they are now fundamentally restructuring computing infrastructure — from the smallest components to massive server farms — to meet the unprecedented (and, pessimistically, untenable) demands of artificial intelligence.

At the heart of this computing paradigm shift is specialized hardware: Graphics processing units (GPUs) originally designed for video games are now the cornerstone of AI development. Unlike conventional central processing units (CPUs) that execute calculations sequentially, GPUs can perform thousands of calculations simultaneously, making it ideal for the mathematical requirements of neural networks that power today’s AI systems.

The race to build more powerful AI requires not just different chips, but entirely new data center architectures. Companies are packing GPUs as densely as possible, specialized with hardware and cabling designed to rapidly stream data between chips. “Everything must function as one giant, data-center-sized supercomputer,” said Meta’s data center VP Rachel Peterson. “That is a whole different equation.” In 2022, the company broke ground on data centers specifically designed for AI training, and in 2023, they spent USD 4.2 bn on restructuring, much of which going towards redesigning older data centers for AI compatibility.

The hunger for power: The industry’s most pressing challenge may be securing enough electricity to power these computing behemoths. AI data centers consume vastly more power than their traditional counterparts — a traditional CPU needs about 250 to 500 watts of electricity, while GPUs use up to 1k watts each. The 5 megawatts that once powered an entire data center can now run just 8-10 rows of GPU-packed computers. As a more relatable comparison, OpenAI’s planned facilities would consume the amount of electricity equivalent to 3 mn households.

“Conversations have gone from ‘Where can we get some state-of-the-art chips?’ to ‘Where can we get some electrical power?’” notes David Katz, operating partner at venture capital firm Radical Ventures. Other companies seem to have similar concerns, and have been trying to work around them — OpenAI backer Microsoft is restarting the Three Mile Island nuclear plant, Elon Musk is installing gas turbines for an xAI data center, and Google and Amazon are both separately exploring nuclear reactor technologies.

The extraordinary heat generated by densely packed AI systems has forced another major innovation: water cooling. Traditional air cooling systems simply cannot keep up with GPU demand. At one data center cited by the New York Times, the air temperature has been seen to increase from 22°C to 36°C after passing through just one rack of AI hardware. To prevent overheating and potential fires, companies are developing sophisticated water cooling technologies. Instead of simply running water pipes through aisles to cool the surrounding air, Google’s Oklahoma campus positions the pipes directly along the side of the chips. The water is chemically treated to reduce electrical conductivity, minimizing risks from potential leaks.

But this cooling process is heating up the planet. In 2023, Google data centers consumed 27.7 bn liters of water, a 17% increase from 2022. In the drought- and fire-prone California, more than 250 data centers consume tns of liters annually. But the environmental impact begins even before the GPUs are installed — manufacturing the processors requires significantly more energy than producing conventional CPUs due to a more complex fabrication process. Its carbon footprint is extended by emissions related to material and product transportation, and because the raw materials needed involve dirty mining procedures and the use of toxic chemicals during processing. “The industry is on an unsustainable path,” says Noman Bashir, Computing and Climate Impact Fellow at MIT. “But there are ways to encourage responsible development of [AI] that supports environmental objectives.”

Tags: