Next-Gen Data Center Fabric: Transitioning from EVPN-VXLAN to End-to-End SRv6

Over the past decade of cloud evolution, EVPN-VXLAN served as the standard foundation for data center virtualization and multi-tenancy. It broke the 4,096 VLAN isolation limit, building a seamless Layer 2 Overlay network over a Layer 3 physical Underlay.

However, with the explosion of distributed Large Language Model (LLM) training, high-density GPU clusters, and cross-data-center collaborative computing, network traffic profiles have fundamentally changed. The legacy VXLAN architecture now reveals structural bottlenecks when handling massive elephant flows, cross-domain protocol translation, and encapsulation overhead. Native IPv6-based Segment Routing (SRv6) is rapidly transitioning from WAN backbones into the data center core, becoming the foundation for next-generation compute Fabrics.

1. Architectural Bottlenecks: Why VXLAN Falls Short in the AI and Multi-Cloud Era

While VXLAN effectively addressed multi-tenant isolation for virtual machines and containers in general cloud environments, its limitations become clear as infrastructure evolves toward heterogeneous compute and multi-center orchestration:

1.1 Cross-Domain "Protocol Stitching" and Gateway Bottlenecks

In Multi-Data-Center (DCI) or hybrid cloud topologies, EVPN-VXLAN typically handles intra-DC traffic, while MPLS, SR-MPLS, or pure IP routing runs across the WAN backbone.

• State Overhead & Single-Point Bottlenecks: Data Center Interconnect (DCI) or Border Leaf gateways must manage complex protocol translation and state maintenance—decapsulating VXLAN, stripping inner labels, querying routing tables, and re-encapsulation into MPLS/WAN packets.

• Operational Fragmentation: Intra-DC and inter-DC networks utilize disjointed OAM monitoring mechanisms, making end-to-end fault boundary isolation and performance telemetry extremely difficult.

1.2 Compute "Elephant Flows," ECMP Polarization, and Congestion

AI compute clusters (e.g., LLM pre-training and fine-tuning) rely heavily on large-scale collective communications (AllReduce, All-to-All). This traffic profile features low flow counts but massive, bursty single-flow throughput (elephant flows).

• Hash Collisions: Traditional data centers rely on Equal-Cost Multi-Path (ECMP) routing for load distribution. Because VXLAN relies on outer UDP source ports for hashing, bursty elephant flows frequently hash to the same physical link, causing severe localized congestion.

• PFC Storms & Reduced MFU: In RoCEv2 lossless networks, link congestion triggers Priority Flow Control (PFC) backpressure. This can cause PFC deadlocks or excessive queuing latency, ultimately degrading Model FLOPs Utilization (MFU) across the compute cluster. VXLAN lacks flexible Traffic Engineering (TE) capabilities to dynamically re-route around congestion.

3.3 Encapsulation Overhead and ASIC Pipeline Parsing Costs

Standard VXLAN adds at least 50 bytes of encapsulation overhead (Outer MAC + IPv4 + UDP + VXLAN Header). During gradient synchronization and frequent small-parameter exchanges in AI workloads, this overhead reduces payload efficiency and places extra load on switch ASIC parsing depth and lookup pipelines.

2. Technical Decoupling: Principles of Next-Generation SRv6 Fabric

SRv6 is not merely a tunneling protocol; it is a network-programmable instruction set built on source routing principles and native IPv6 extension headers.

2.1 Unified Addressing & Network Instruction Sets: Locator:Function:Args

SRv6 abstracts network nodes and service behaviors into 128-bit IPv6 addresses (Segment IDs, or SIDs) split into three logical fields:

• Locator: Guarantees network-wide Underlay reachability. Intermediate nodes perform Longest Prefix Matching (LPM) using standard IPv6 routing (e.g., BGP/OSPFv3) without maintaining complex tunnel tables.

• Function: Specifies the execution instruction at the destination node.

  • End.DT4 / End.DT6: Decapsulate and perform an IPv4/IPv6 table lookup in a designated VRF (Layer 3 multi-tenancy).

  • End.DX2: Decapsulate and forward the inner Layer 2 frame to a designated interface (Layer 2 EVPN service).

• Args (Arguments): Carries flow-matching, security metadata, or network telemetry data.

2.2 EVPN over SRv6: Flat Multi-Tenancy Delivery

The control plane leverages established BGP EVPN with simplified mechanics: During route advertisement, the Egress Leaf advertises service SIDs (e.g., 2001:db8:1::End.DT4) to the network via the BGP Prefix-SID attribute. Upon receiving tenant data, the Ingress Leaf encapsulates the packet with the corresponding IPv6 SID as the Destination IP. Packets route transparently across the Fabric without requiring Spine nodes to maintain tenant isolation state.

2.3 SRv6 Policy & Intelligent Source Routing

To address RoCEv2 elephant flow congestion in AI clusters, SRv6 provides deterministic traffic orchestration:

• Explicit Path Selection: The Ingress Leaf or a centralized controller pushes a Segment List into packet headers based on real-time telemetry, routing around congested links.

• Stateless Intermediate Forwarding: Intermediate Spine switches forward traffic based on the active SID and decrement the pointer (Segments Left minus 1), eliminating granular flow table provisioning on core switches.

2.4 SID Compression: Resolving MTU Overhead

To address packet expansion caused by standard 128-bit Segment Routing Headers (SRH), RFC 9800 defines two compression mechanisms: NEXT-C-SID and REPLACE-C-SID. By packing multiple short opcodes (16-bit or 32-bit) into a single 128-bit IPv6 destination container, packet overhead is minimized while maintaining standard throughput for small packets and ASIC lookup efficiency.

3. Typical SRv6 Deployment Scenarios in Data Centers

3.1 SRv6 in AI Backend Networks — MRC Architecture

The Multi-Rail Carrier (MRC) architecture—developed collaboratively by OpenAI, Microsoft, NVIDIA, AMD, Intel, and Broadcom—demonstrates SRv6 in AI backends. A key principle of SRv6 is giving applications direct control over their network path. Implementing MRC at the transport layer creates a programmable fabric where the transport stack selects paths on a per-packet basis. Sprinkling packets across stateless paths and planes avoids low-entropy flow collisions common in traditional ECMP setups.

3.2 SRv6 Container Networking — NetPila

Alibaba Cloud's NetPila illustrates how SRv6 improves container networking beyond VXLAN tunnels. Embedding tenant and interface identifiers directly into the IPv6/SRv6 address model enables tunnel-less Pod-to-Pod connectivity and tight endpoint-network integration.

3.3 SRv6 DCI & Backbone Networking — eCore

Alibaba Cloud's eCore is an IPv6/SRv6 backbone and DCI architecture. Utilizing a unified IPv6 Underlay and SRv6 Traffic Engineering, eCore shifts path selection from hop-by-hop decisions to programmable, end-to-end control. In AI deployments, server-side applications set Flow Labels based on workload properties and QoS requirements, which the network maps to specific SRv6 SID-Lists for optimized path selection (e.g., RDMA, high-bandwidth, or low-latency links).

3.4 SRv6 Data Center Frontend Networks

• Eliminating DCI Bottlenecks: Removes EVPN-to-MPLS translation logic on DCI gateways. DCI devices function as high-capacity IPv6 routers, avoiding multi-vendor interoperability issues.

• Simplified NFV Service Chaining: NFV instances (e.g., Cloud Gateways, Firewalls) running on x86 hosts natively process IPv6/SRv6 in kernel, replacing hardcoded routes with flexible instruction sequences.

• Unified Control: Replaces multi-segment "DC + DCI + WAN" designs with an end-to-end, flat L3VPN architecture.

3.5 SRv6 Service Chaining

1. Services as SIDs: In VXLAN networks, service nodes (such as firewalls) act as topological "black holes" requiring static policy routing. In SRv6, every service node or interface receives a globally unique SID. Service chains append a Segment List of service SIDs to the packet header, allowing automatic network orchestration.

2. Unified Control and Forwarding: VXLAN Overlay networks require separate control planes (e.g., EVPN) and controller-driven service policies. SRv6 unifies control (IGP/BGP SID advertisements) and data planes (SID encapsulation), simplifying the stack.

3. Stateless Service Chaining: Traditional service chains require state maintenance across service nodes. SRv6 service chaining is completely stateless—pathing and service ordering are driven entirely by the packet's SID list.

4. AsterNOS SRv6 Specifications and Roadmap

To support end-to-end SRv6 Fabric deployments, AsterNOS provides a comprehensive feature roadmap. AsterNOS supports standard SRv6 and REPLACE-C-SID (G-SID) compression, with full NEXT-C-SID (uSID) support arriving in Q4 to assist enterprise AI networks.