Subscribe and receive the latest news from the industry.
Join 56,000+ members. Yes it's completely free.
AI Rewrites the Rules for Connecting the Enterprise Edge
AI-driven networking reshapes edge requirements, creating new opportunities for CSPs.
November 22, 2024
8 Min Read
iStock
Sponsored by VMware
In our role as the leading provider of software-defined wide-area networks (SD-WAN), with more than half a million VeloCloud SD-WAN Edges deployed worldwide, Broadcom has a front-row seat to the ongoing evolution of enterprise networks. Over the past year, we’ve tracked one of the most consequential shifts in the application landscape we’ve ever seen: AI.
Artificial intelligence is revolutionizing business operations with applications like chatbots, video inferencing, co-pilot systems, and many others. Now, organizations are starting to deploy the next wave of AI applications, such as retrieval augmented generation (RAG)-based systems and agentic AI, which can apply even more advanced intelligence and real-time decision-making to practically every part of the business.
As these changes ripple through the enterprise, however, many organizations find themselves bumping up against a major barrier: current edge networks. AI applications introduce radically different networking requirements than most enterprises have dealt with before. In edge networks in particular, AI workloads disrupt multiple long-held assumptions about how these networks should be architected—upstream/downstream traffic ratios, acceptable latencies, encrypted traffic-handling, and others. In many cases, existing edge networks simply can’t meet these changing needs.
Why do AI workloads place such different requirements on enterprise networks? What do organizations need from edge environments to unleash AI innovation? And how are these factors converging to create a growing business opportunity for communication service providers (CSPs)? Let’s take a closer look.
What makes AI different?
For a typical enterprise network, generative AI (GenAI) applications introduce not only different traffic patterns but different ways of using network resources. These applications are bursty, highly sensitive to latency, and functionally operate on a peer-to-peer basis. They upend conventional upstream/downstream traffic patterns. And they use encrypted flows that can’t easily be analyzed or optimized, making WAN traffic far less predictable. These differences introduce new constraints and requirements for edge networks across multiple dimensions, including:
Symmetrical traffic ratios: Traditional WAN circuits skew heavily towards download traffic. Streaming video, for instance, has a download-to-upload ratio of 99:1. While some services use more upstream bandwidth, most edge networks were designed for conventional web applications, which average about 85:1. GenAI application traffic looks very different, typically seeing higher volumes of both upload and download traffic with much larger request sizes. AI applications also typically incorporate multimodal data (including images and video) for analysis and inferencing, routinely pushing the ratio to 50:50. Emerging applications, like Meta’s AI-enabled Ray-Ban smart glasses, generate 99 times more upstream traffic than downstream—a ratio that few existing edges were architected to support.
Higher throughput: One of the key mechanisms organizations use to reduce traffic loads at both data centers and distributed sites is content delivery networks (CDNs). GenAI traffic, however, is typically unique and personalized for each user, requiring much more computational overhead and bandwidth, and making it much more difficult to cache. As a result, WAN circuits and supporting infrastructure connecting into the data center must provide significantly higher bandwidth.
Low latency: Any real-time application will be sensitive to latency, but for use cases like AI-enabled video monitoring, behavioral analytics, or augmented reality, even small amounts of latency can break the application experience. As GenAI architectures evolve to become more distributed, with large language models (LLMs) running centrally and one or more small language models (SLMs) at the edge—all employing multiple interactions and inputs from multiple sources—the effects of latency become even more pronounced.
Bursty, unpredictable traffic patterns: Most traditional web applications have highly predictable access patterns. GenAI interactions, however, typically involve high traffic while the client uploads a large request, followed by a lull while the request is processed, then a large response sent back. This variability means that, to assure good quality of experience (QoE) for users, the underlying network must be able to adapt and shape traffic in near real time as traffic patterns change. This requires an extraordinary degree of agility from edge networks, ideally including the ability to make steering decisions at a per-packet level and decouple logical application flows from the underlying network.
Peer-to-peer access: Unlike traditional web applications, which typically have client-server access patterns, AI application architectures tend to be highly distributed, with many peers interacting simultaneously. A GenAI application might involve one or more LLMs running in public cloud or large private data centers interacting with multiple SLM agents at the edge. A single request can involve multiple interactions between these components, with large volumes of private user data and proprietary model data flowing back and forth among peers. This traffic is also typically encrypted, making it even harder for traditional edge environments to shape and prioritize traffic flows.
That’s a long list of changes that most current edge networks were never designed to support. Indeed, until recently, many organizations assumed that the application stack of the future would largely live in the cloud, with distributed locations consuming far more traffic from data centers than they generated. AI turns that assumption on its head, and much of the existing infrastructure at distributed enterprise locations will need to be re-architected.
Envisioning the AI-Ready Edge
Meeting the requirements of AI applications in edge networks requires more than just additional bandwidth. An intelligent software layer becomes essential—one that can understand the needs of different AI application workloads and dynamically adjust network resources to prioritize business-critical applications and maintain QoE for users. It’s the only way to address the unpredictable traffic patterns and requirements of the highly variable, rapidly growing body of potential AI use cases.
The good news is that we already have an enormously successful model for applying sophisticated traffic-handling intelligence at the edge: SD-WAN. Indeed, it’s hard to imagine how many of the distributed AI applications that enterprises are now contemplating will be possible without SD-WAN intelligence. An AI-optimized SD-WAN should provide capabilities like:
Bandwidth optimization, using techniques like link bonding to pool uplink bandwidth on demand and increase uplink capacity at the infrastructure level, independent of the application
AI-enabled traffic prioritization, tracking latency for each application and making steering decisions to prioritize more delay-sensitive AI applications and maximize QoE
Near-real-time traffic adaptation and shaping, at a per-packet level, to support bursty traffic patterns as they change
Recognizing that AI applications effectively create a distributed peer-to-peer mesh, an AI-ready SD-WAN should also provide an overlay construct to securely connect peers across locations and encrypt their interactions. It should bring together diverse underlying WAN links, so it can support different locations with different performance characteristics while maintaining consistent QoE.
Introducing VeloRAIN
As the world’s premier SD-WAN provider, VeloCloud has been working diligently to address these changing edge requirements and help unleash a new generation of enterprise AI applications. We recently unveiled the fruits of these efforts with VeloCloud Robust AI Networking, or VeloRAIN.
VeloRAIN builds on our leading VeloCloud SD-WAN platform, providing an AI networking architecture that enhances the performance, security, and scalability of AI workloads across distributed enterprise networks. It does this by not only adding new capabilities to support distributed AI applications, but by incorporating advanced AI intelligence itself.
VeloRAIN builds on VeloCloud’s patented Dynamic Multipath Optimization (DMPO) intelligence to continually analyze and optimize application performance in real time. It uses machine learning enhancements to accurately profile applications — including identifying encrypted AI traffic so it can be distinguished and prioritized— and ensure that each AI and traditional application has the network resources it needs. Additionally, VeloRAIN adds dynamic application-based slicing (DABS), a new approach to assure per-application (and even per-user) QoE across multiple disparate underlying networks.
Drawing on anonymized data from our vast deployment base, VeloRAIN also uses AI to automate network operations (AIOps) and dynamically adjust policies in real time. This is especially important for high-bandwidth, latency-sensitive applications like real-time video analytics and will allow enterprises to adjust network configurations on-the-fly to ensure the best possible user experience.
Finally, we’re using AI to enhance security at the edge. The explosive growth of AI understandably has enterprise security leaders worried about the risk of users releasing unauthorized bots and AI services inside their networks, without their knowledge. With VeloRAIN enabling virtual segmentation of peer-to-peer AI traffic, enterprises can isolate agentic and other AI traffic and safeguard against new types of threats.
A Growing Telecom Opportunity
As enterprises reimagine their networks for AI, these changes will have far-reaching consequences across the digital ecosystem—nowhere more so than for CSPs. As the enterprise’s trusted provider of WAN connectivity, and a longtime partner for SD-WAN solutions as well, CSPs have a central role to play in enabling edge transformation.
In the first instance, enterprises expanding their AI footprint will need more bandwidth at the edge, especially upstream. But they’ll also need expert guidance to sort through the dizzying range of potential AI-enabled applications and use cases that different organizations might employ, each with its own unique requirements. Effectively, enterprises need in-depth consultative assistance to secure the right connectivity and SD-WAN solution for each location. No one is better positioned than CSPs to provide that guidance, even as they increase enterprise connectivity revenues and layer on higher-margin services like managed security and SD-WAN.
We’re still in early days of the AI revolution, and no one can say definitively which applications will prove most transformative, or what tomorrow’s business-critical edge workloads will require. But with a new generation of AI-ready SD-WAN technologies, and expert CSP partners ready to help enterprises make the most of them, the future of AI innovation looks bright.
Read more about:
Vendor SpotlightsYou May Also Like
_1.jpg?width=300&auto=webp&quality=80&disable=upscale)
