Glossary

ARS

Adaptive Routing and Switching

What is ARS

ARS (Adaptive Routing and Switching, referred to as Adaptive Routing Switch in some AsterNOS configuration documentation) is a flowlet-based load balancing technology for data center fabrics. Instead of locking each flow to a single path for its entire lifetime, ARS breaks a flow into smaller bursts called flowlets and steers each flowlet onto the path that is currently least congested. The result is traffic distribution that approaches the evenness of per-packet load balancing, while avoiding most of the packet reordering that per-packet methods cause.

ARS exists because the standard approach to load balancing in data center networks — ECMP (Equal-Cost Multi-Path) per-flow hashing — struggles with the traffic that AI workloads generate. ECMP assigns packets to paths by hashing fields such as the IP 5-tuple, which is a form of static load balancing. When traffic has low entropy, meaning few distinct flows or flows that look alike to the hash, distribution becomes uneven, and large "elephant flows" can overwhelm individual member links and cause packet loss. AI training makes this worse: collective communication operations such as All-Reduce, All-Gather, and Broadcast produce massive, bursty traffic that can reach terabits per second, and they are subject to the "straggler effect," where congestion on a single link holds back an entire training job. Static per-flow hashing is a poor fit for these conditions, and ARS is designed to replace it in RoCEv2-based AI backend fabrics.

How ARS Works

ARS relies on the hardware auto-load-balancing (ALB) capabilities of the switching silicon, and its behavior can be understood in three parts.

Flowlets. A flowlet is a burst of packets from a flow that is separated from the next burst by a gap in time. ARS divides each flow into a series of flowlets based on this gap: whenever the inter-frame gap within a flow exceeds a configured interval, ARS treats the next packets as a new flowlet and is free to place it on a different path. Because packets inside a flowlet always follow the same path, and the gap between flowlets gives earlier packets time to clear the network, reordering is largely avoided.

Quality-aware path selection. For each new flowlet, ARS senses real-time link quality, such as bandwidth utilization and queue depth, and assigns the flowlet to the most idle path available. This is what makes the routing "adaptive": placement decisions follow actual conditions on the fabric rather than a fixed hash, which maximizes overall throughput across the available links.

Idle-time. The gap that triggers a flow split is called the idle-time, and it controls how finely a flow is divided. The recommended starting point is half the network's round-trip time (RTT/2), beginning from the system default and adjusting based on observed traffic:

  • If significant packet reordering is detected at the endpoints, increase the idle-time, so flows split less often.

  • If load distribution between the Leaf and Spine layers looks unbalanced, decrease the idle-time, so flows split more often and placement can correct itself sooner.

Instances and next-hop groups. In a Clos fabric, ARS is organized as instances that map one-to-one onto next-hop groups (ECMP groups). The number of instances needed differs by layer:

  • On a Leaf switch, every route advertised by the other Leafs shares the same ECMP members — the uplinks to the Spines — so a Leaf needs only a single ARS instance to manage all northbound traffic.

  • On a Spine switch, each Leaf advertises its own routes, and the ECMP group for a given Leaf consists of the physical links from that Spine to that Leaf. A Spine therefore needs one ARS instance per Leaf, so the number of instances on a Spine must equal the number of Leaf switches.

Each instance is then bound to its destination by specifying the Loopback IP of the relevant Leaf. On a Spine, binding to Leaf1's Loopback IP ties that instance to the links toward Leaf1; on a Leaf, specifying the Loopback IP of any other Leaf binds traffic to the instance covering the uplinks to all Spines.

Hash seed and polarization. ARS is often deployed alongside a related fix for hash polarization, an imbalance that appears in multi-tier Clos topologies when every tier uses the same ASIC and therefore the same default hash algorithm. The second tier then fails to redistribute traffic that the first tier already hashed, leaving some links overloaded while others sit underused. Setting a different hash seed on devices at different layers ensures each stage hashes differently and restores a more even spread.

Why ARS is Beneficial

  • Distributes traffic by actual conditions, not a fixed hash: Placement decisions use live bandwidth and queue-depth information, so traffic flows toward idle paths rather than piling onto whichever links the hash happened to select.

  • Handles elephant flows and low-entropy traffic: Splitting a large flow into flowlets lets it use several paths over time, which static per-flow ECMP cannot do.

  • Keeps reordering under control: Placing traffic at flowlet boundaries gives near per-packet balance without the heavy reordering of per-packet spraying, which otherwise depends on high-performance NICs capable of out-of-order reassembly at the endpoint.

  • Reduces the straggler effect: By spreading bursty collective-communication traffic evenly across the fabric, ARS lowers the chance that one congested link delays an entire training job.

  • Tunable to the environment: The idle-time setting lets operators trade off between balance and ordering, based on measured behavior.

  • Complements lossless networking: ARS balances load across paths, while ECN and PFC manage congestion within each path, so the pieces work together in a lossless RoCEv2 fabric.

At Asteraix

What We Can Do at Asteraix

AsterNOS provides ARS as a hardware-accelerated feature on its data center switches, configured through a short, structured workflow: create ARS instances, bind next-hop groups, then fine-tune idle-time.

  • Simple, three-step configuration: ars profile enables ARS; ars instance <name> followed by idle-time <value> creates an instance with its idle-time; and ars nexthop-group <Loopback-IP>/32 instance <name> binds a destination to that instance. AsterNOS's documented example uses an idle-time of 10 as a starting point.

  • Layer-aware design built in: In the documented CX864E-N reference design, each Leaf uses a single instance (for example, to_spine) bound to another Leaf's Loopback address, while each Spine defines one instance per Leaf (to_leaf1 through to_leaf4) bound to the matching Leaf Loopback addresses — exactly mirroring the one-to-one instance and next-hop-group relationship described above.

  • Hash seed configuration to prevent polarization: hash seed <value> — set to 1234 on the Spines in the documented example — gives operators a direct way to differentiate hashing between layers of a Clos fabric.

  • Works with Easy RoCE: ARS is deployed together with Easy RoCE (qos roce lossless cable-length <length> incast-level <level> traffic-model <model>), which automates the lossless PFC and ECN configuration, so load balancing and congestion control come as a unified solution across both Leaf and Spine nodes.

  • Built for large AI backend fabrics: The reference design uses CX864E-N switches — 64 × 800G ports that can be split into 128 × 400G, with end-to-end latency as low as 560ns — in a 2-tier, Rail-optimized Clos network with unnumbered BGP over IPv6 link-local addressing and a 1:1 oversubscription ratio, and scales to thousands of GPUs.

  • Clear verification commands: show ars profile displays the ARS profile configuration, and show ars instance lists each instance with its name, assign mode (per-flowlet quality), idle time, binding configuration, and — once routes are reachable — the actual next-hop group members and member count.