Artificial intelligence is a transformative force that is changing the future of technology in general, and data centers in particular. This blog examines the increasing data center power density demands driven by AI.
The Dawning of the Age of AI
While it's true that AI has been around for a while, we are at an inflection point that could be considered the dawn of the age of AI. Innovators have only started to explore the many ways AI can support, improve and radically alter every business vertical.
“AI's transformational potential is analogous to paradigm-shifting technologies like the advent of the internet or the mobile phone, and it has the potential to touch every sector of global industry,” according to the 2023 U.S. Data Center Market Overview from Newmark.1
The diversity of use cases is one of the attractions making AI so popular. A 2024 report from Barclays elaborates, “These tools are not confined to any particular task, function, problem or sector. This makes them usable across different disciplines. Once a Large Language Model (LLM) is trained on a body of text, for example, it can summarize a legal document as well as it can a medical or insurance document.”2
Due to this wide appeal, AI adoption rates are high. McKinsey & Company found that 72% of organizations use generative AI (GenAI)3, an increasingly popular form of AI. And 90% of organizations are investing in GenAI, according to a report from Dataiku and Databricks.4
A majority of respondents (56%) in a 2024 AFCOM survey plan to deploy AI-capable solutions in their data centers.5 This is having a dramatic impact on data center power densities.
The Power of AI Depends on Power
AI – especially AI model training – is much more energy-intensive than the applications that drove data center growth over the past two decades, according to a report from energy R&D specialist EPRI.6
For example, on average, a ChatGPT query uses nearly 10 times as much electricity to process as a Google search. “In that difference lies a coming sea change in how the world will consume power,” Goldman Sachs predicts.7
AI data centers use graphics processing unit (GPU) clusters instead of traditional central processing unit (CPU) clusters. The GPUs designed to handle AI, and perform complex calculations 24 hours per day, require enormous amounts of power. “The power usage of a standard processing unit, or chip, inside a server is around 75-200 watts, similar to a light bulb,” explains Shin Gomi, Senior GM leading the data center and energy management business at Mitsubishi Heavy Industries. “But one of Nvidia’s GPU chips essential to AI applications needs 5-10 times as much power and will generate 5-10 times as much heat. This is expected to drive the energy consumption of a typical hyperscale data center from 30-40 Megawatts currently to over 100MW in three to five years’ time.”8
“The average rack power density today is around 15 kW/rack, but AI workloads will dictate 60–120 kW/rack to support accelerated servers in close proximity,” adds Lucas Beran, Research Director at Dell’Oro Group.9
Also important to note, AI is not the only emerging high-performance computing technology that requires more power density in the data center. Cryptocurrency mining, video streaming, data analytics, and the Internet-of-Things (IoT) are all contributing to the data center power density challenge as well.10
AI Makes Liquid Cooling Essential
According to the International Energy Agency (IEA) most of the power consumed in a data center comes from computing (40%) and cooling (40%)11 – and both will be exacerbated by AI. While greater computing power in the data center translates directly to greater power density, AI also drives up power density because the dense clusters of high-performance chips generate more heat than typical processing units – and the cooling needed to counteract the rising temperatures and keep the hardware from overheating also requires more power. That is why more energy-efficient data center cooling solutions are an important factor in the AI equation.
“Data center capacity for GPU-based AI computing must expand rapidly, which is driving innovations like liquid cooling,” says the Newmark data center report.1 Liquid cooling is an innovative solution that is highly effective and requires less power than conventional air cooling. Experts believe that the increasing power densities driven by AI – requiring 60kW or more per rack – make liquid cooling an absolute necessity.
This is one of the reasons that liquid cooling is gaining greater adoption across the data center industry. AFCOM says that respondents are turning to liquid cooling for help to support emerging cooling requirements. “Today, only 17% of respondents have adopted liquid cooling in their facility. However, an additional third (32%) plan to do so within 12-24 months,” AFCOM reports.5 Many consider liquid cooling to be the future of data center cooling.12
Colocation: The AI Data Center Solution
Heightened expectations around AI, especially GenAI, are causing IT leaders to reevaluate their options for hosting these and other high-density workloads to meet physical infrastructure (power and cooling) and data management requirements within budget constraints, according to CoreSite's 2024 State of the Data Center Report.13
Colocation is a powerful alternative. A 2023 IDC study found that colocation is the preferred model for deploying enterprise AI and ML models.14 CoreSite's 2024 State of the Data Center report confirms this finding, as three-quarters of respondents are considering moving AI-related workloads from the public cloud to a colocation data center.13
“Organizations of all sizes, from SMBs to Fortune 500 enterprises, are eager to embark on an AI journey ─ and colocation is a key enabler,” says Bobby Rogers, VP of Sales of Strategic Accounts for Schneider Electric’s Data Center Solutions Team.15
Know More
CoreSite data centers are designed to be AI-enablers, where customers can develop and execute forward-thinking AI strategies. CoreSite is certified by the NVIDIA DGXTM-Ready Data Center Program, and offers the power density, energy-efficient designs, liquid cooling solutions, and robust network interconnections needed for AI and other high-performance workloads.
Download the position paper, "Trust CoreSite Data Centers to Enable Your AI Strategy" to learn more about how CoreSite colocation data centers can help transform your business into a future-ready enterprise.
References linked here