Why the Network Matters to Generative AI

If you studied computer science, whether undergrad or beyond, then you’ve probably taken a course in computer architecture. Not the kind we draw on diagrams today illustrating a data center architecture, but the deep-down-at-the-circuit kind of architecture.

You know, where buses connect components like CPU, ALU, RAM, and, of late, GPU and DPU. Design of these systems requires answering questions about how fast the bus speeds between components must be and how much bandwidth is required to support a given set of performance requirements. This is where technologies like I²C, PCI, and QPI fit, why FSB is no longer used, and why DDR replaced SDR. The “network” that connects circuit-level components is a significant factor in processing speed and capacity.

If we take a step back and critically examine the data center architecture, we see it's a larger version of the same architecture that requires us to answer the same questions. The need for speed, increased bandwidth, and very low latency are why we're seeing AI compute complexes leverage alternate networking technologies like InfiniBand. The network matters, you see. 

Now back up one more step and look at all this from a global level, where clouds and data centers are the components, and the Internet are those buses. 

A look back to look forward

Patterns, they say, repeat. In the world of architecture, they repeat at multiple scales, like fractals. This is not a new idea, as the “father of fractals” observed long ago: 

"Fractal geometry provides a framework for understanding complex network structures by revealing how self-similar patterns can emerge at different scales."

- Benoît B. Mandelbrot

Now, good architects excel at abstracting patterns and leveraging their strengths at every level. When one jokingly says, “The Internet is my computer,” they’re kind of not joking. From a high enough perspective, it really is just a ginormous, distributed computer today.  

So, it will not be a surprise when I point out the importance of the network to such a distributed computing complex. The speed, security, and paths across that network matter to the performance and capacity of whatever instruction—API calls between applications—is making its way to the right component for execution.

Applications, today, are distributed. Our core research tells us more than half (60%) of organizations operate hybrid applications; that is, with components deployed in core, cloud, and edge locations. That makes the Internet their network, and the lifeline upon which they depend for speed and, ultimately, security.

Furthermore, our focused research tells us that organizations are already multi-model, on average deploying 2.9 models. And where are those models going? Just over one-third (35%) are deploying in both public cloud and on-premises.

Applications that use those models, of course, are being distributed in both environments. According to Red Hat, some of those models are being used to facilitate the modernization of legacy applications. Legacy apps are typically on-premises, even if the AI used to modernize is it somewhere else.

The role of multi-cloud networking

So, we’ve got applications and AI distributed across the Internet, and a network that needs to connect them. Oh, and it’s got to be secure as well as fast.

This is why we're seeing so much activity focused on multi-cloud networking solutions. The misnamed technology trend (it’s not just about multiple clouds but about interconnecting multiple locations) is a focus on the network and a recognition of the important role it plays in securing and delivering applications today.

One is likely tempted to ask why we need such a thing. The problem is we can’t affect the Internet. Not really. For all our attempts to use QoS to prioritize traffic and carefully select the right provider, who has all the right peering points, we can’t really do much about it.

For one thing, over-the-Internet connectivity doesn’t typically reach into another environment, in which there are all kinds of network challenges like overlapping IP addresses, not to mention the difficulty in standardizing security policies and monitoring network activity.

These are the problems multi-cloud networking solves for. Multi-cloud networking basically extends a network into multiple environments rather than just connecting those environments via two secure endpoints, a la a VPN.

Multi-cloud networking is becoming increasingly important to the success of generative AI because the architectural pattern—whether at the board or application layer—always depends on the ability to transfer data between components safely, reliably, and as fast as possible. Multi-cloud networking introduces some of the control network professionals are missing when they have to use the Internet as their network.  

Related articles: