RDMA

RDMA (Remote Direct Memory Access) is a networking technology that allows one computer to directly read from or write to the memory of another computer — bypassing the operating system kernel, CPU, and intermediate buffers entirely. The result is ultra-low latency, high throughput data transfer that has become foundational to AI training clusters, high-performance computing, and large-scale data centers.

Why RDMA Emerged

As AI model training, high-performance computing (HPC), and hyperscale data centers have grown in scale, the volume of data exchanged between server nodes has increased exponentially. Traditional TCP/IP networking requires multiple context switches between kernel and user space, intermediate memory copies, and CPU interrupts to handle each data transfer — overhead that becomes a serious bottleneck when workloads operate at microsecond timescales.

RDMA was conceived in the early 1990s alongside the InfiniBand specification, with the goal of eliminating this overhead through “zero-copy” and “kernel bypass” data movement. Today, with the maturation of RoCE (RDMA over Converged Ethernet), the benefits of RDMA have extended to standard Ethernet infrastructure — making it a standard feature of AI clusters, storage fabrics, and cloud platforms worldwide.

How DMA and RDMA Work

To understand RDMA, it helps to start with its predecessor: DMA (Direct Memory Access). In traditional computer architecture, the CPU mediates all data movement — a process that consumes significant compute resources. DMA allows peripherals such as network cards and disk controllers to exchange data directly with system memory without CPU involvement, freeing the processor to focus on computation.

RDMA extends the DMA concept across a network. It allows a server’s RDMA-capable network interface card (RNIC) to read from or write to the memory of a remote server, without that remote server’s CPU or operating system being involved. The process works as follows:

• Sender: The application registers a memory region. The RNIC reads data directly from that region and transmits it — no kernel involvement required.
• Network transit: Data packets are forwarded at high speed by RDMA-aware switches, without per-hop unpacking and reassembly.
• Receiver: The RNIC writes data directly into the target memory address and notifies the application that the operation is complete — the CPU is never involved.

This “zero-copy, kernel-bypass” mechanism is the fundamental reason RDMA achieves microsecond-level latency and throughput exceeding hundreds of gigabits per second.

Three Implementations of RDMA

Native RDMA — InfiniBand A purpose-built, proprietary network architecture designed for HPC with native RDMA support. It delivers the lowest latency and highest bandwidth available — up to 800G NDR today — but requires dedicated HCA adapters and InfiniBand switches. The preferred choice for supercomputing centers and large-scale AI training clusters where performance is paramount.

Ethernet RDMA — RoCE (RoCEv2) Brings RDMA capabilities to standard Ethernet networks. RoCEv2 runs over UDP/IP and supports routable, cross-subnet deployment — making it the most widely deployed RDMA solution in enterprise data centers. It reuses existing Ethernet switches but requires lossless network configuration via PFC and ECN flow control mechanisms.

WAN RDMA — iWARP Implements RDMA over TCP/IP, offering the broadest compatibility with existing network infrastructure — no special switch configuration required. TCP’s reliability makes iWARP better suited to environments with packet loss, such as wide-area networks, though it carries slightly higher latency than InfiniBand or RoCEv2.

Core Advantages of RDMA

Through its three foundational characteristics — zero-copy transfer, kernel bypass, and CPU offload — RDMA delivers measurable advantages across multiple dimensions of network performance.

• ⚡Ultra-Low Latency By bypassing the OS network stack and eliminating kernel/user-space context switches, RDMA achieves end-to-end transfer latencies in the single-digit microsecond range — orders of magnitude faster than traditional TCP/IP.
• 🚀High Throughput RDMA supports transfer rates in the hundreds of gigabits per second, meeting the extreme bandwidth demands of AI training, high-frequency trading, distributed storage, and large-scale scientific computing.
• 🖥️CPU Offload Data movement is handled entirely by the RNIC, freeing CPU cores from networking overhead. Those resources can be redirected to business logic and computation, significantly improving overall system efficiency.
• 📋Zero-Copy Transfer Data flows directly from source memory to destination memory with no intermediate kernel buffer copies. This reduces memory bandwidth consumption and eliminates latency jitter caused by memory allocation.
• 🔒Memory Protection RDMA operations require explicit memory registration and access authorization before any transfer can occur. Remote peers can only access explicitly granted memory regions, ensuring strong isolation in multi-tenant environments.

RDMA vs. Traditional TCP/IP Networking

The table below compares the three RDMA implementations against traditional TCP/IP across key dimensions, helping you select the right approach for your workload and infrastructure constraints.