Artificial intelligence (AI) continues to drive transformation across many industries, and as this technology rapidly evolves, so does the infrastructure needed to underpin it. 

Data centers globally are undergoing significant changes to accommodate the power-hungry demands that AI tools place on hardware, networking, energy consumption, and cooling. Some businesses are even building dedicated AI data centers to help fuel the development of their own AI technologies.

So, what exactly does AI technology demand when it comes to data center infrastructure, versus traditional data center workloads?

From CPUs to GPUs

One of the biggest challenges AI causes in a data center environment is its heavy reliance on GPU-based computing. GPUs support AI models by handling many simultaneous calculations. This is vital for facilitating the huge computational needs that come with training and running AI models. Traditional CPUs may be excellent at sequential processing, but as a result, they’re far too slow for many AI models to perform at an optimum level.

All this means that AI data centers must be packed full of GPUs that are significantly more energy intensive since they work at much greater voltages. 

With AI-enabled racks demanding up to six times more power than their traditional counterparts, data center developers are increasingly prioritizing locations where renewable energy is abundant and climates are naturally cooler. Regions in Canada and Iceland are ideal because of the abundance of hydropower and geothermal energy capable of supporting reliable and affordable power for high-density AI workloads. 

However, location is always about striking the right balance. This focus on strategic locations introduces a tradeoff: facilities may be built farther from end users, so any potential impact on latency needs to be taken into account. For some data centers, it’s a case of meeting somewhere in the middle - building in a location with some access to hydropower and a temperate climate, as well as investing in advanced cooling technologies like liquid and direct-to-chip cooling that offer better heat dissipation and greater energy efficiency.

Networking innovations to support evolving AI demands

AI places increased computational demands on servers, as greater quantities of data need to get to and from GPUs as fast as possible. 

AI-driven applications also demand exponentially higher bandwidth to process the immense volumes of data they require efficiently. Servers can need data transfer speeds up to 100Gbps to ensure AI tools and applications run properly. Achieving this will necessitate changes to how providers of GPU-based computing select and build their networking stack. Components that these providers have likely used for years will no longer be sufficient, requiring new selection and R&D processes to take place.

As a result, data center operators are investing in high-performance interconnects that accelerate data transfer between large numbers of computing nodes like GPU clusters and TPUs (Tensor Processing Units), all of which are essential for training and running complex AI models efficiently. The same goes for investments in advanced networking hardware that offers higher throughput, greater reliability, and reduced latency.

The future of the AI data center 

To stay ahead of the game, everyone is trying to catch lightning, and right now, that lightning is AI.  

Technically, AI can be run in any data center. But the greater power and cooling requirements of AI’s GPU-based computing needs mean that not every data center is cost-optimized to run AI. And in an industry where competition is rife, and demands for new AI innovations are high, placing even greater demands on AI workloads based in traditional data centers means that costs can easily spiral. 

Managing these costs needs to be a critical consideration for any data center operator building for AI right now. Whilst many businesses will be willing to pay a premium to run their AI workloads, if data center operators want to remain competitive, they’re going to have to find ways to offset these costs and avoid passing the premium entirely to their client base.