Ultra Ethernet Promises New RDMA Protocol

This week saw the formal launch of the Ultra Ethernet Consortium (UEC), which aims to reinvent Ethernet fabrics for massive-scale AI and HPC deployments. An impressive list of founding members back this ambitious effort: hyperscalers Meta and Microsoft; chip vendors AMD, Broadcom, and Intel; OEMs Arista, Atos, and HPE; and Cisco, which straddles the chip and OEM camps. Absent this backing, we could easily write off this consortium as doomed to failure.

Our skepticism is rooted not in the obvious need the UEC looks to serve but rather in the challenges of standardizing and implementing a full-stack approach. The effort plans to replace existing transport protocols as well as user-space APIs. Specifically, the Ultra Ethernet Transport (UET) protocol will be a new RDMA protocol to replace ROCE, and new APIs will replace the Verbs API from the InfiniBand heritage. UET will provide an alternative to RoCEv2 and Amazon’s SRD, both of which are deployed in hyperscale data centers.

(Source: Ultra Ethernet Consortium)

If the concept sounds familiar, you may have read our recent piece on NVIDIA’s Spectrum-X. That end-to-end Ethernet fabric builds on the ROCEv2 protocol, and it uses DPUs in endpoints to assist in congestion management. The Spectrum-X software stack supports only NVIDIA networking hardware, and the underlying protocol still relies on Priority-Based Flow Control (PFC) as a backstop against packet loss. Microsoft and others have documented the congestion problems that PFC can cause.

The UEC laid out some of the features planned for its new transport protocol. Some key new features include multipath packet spraying, out-of-order packet delivery, and novel rate-control algorithms. Given its early involvement, we expect Broadcom will contribute intellectual property related to the edge-queued datagram service (EQDS) described in an NSDI 2022 paper. Last year, it quietly acquired Correct Networks, which developed EQDS. An optional EQDS element is packet trimming, which drops payloads but delivers headers to inform receivers of network congestion. EQDS employs tunnels, however, rather than replacing the RDMA transport protocol.

To support the UET, the consortium will define new APIs that enable out-of-order packet delivery. Beyond the transport layer, the UEC will also address link-layer issues such as forward-error correction and retry mechanisms. These efforts may feed into other organizations including IEEE 802. One of the reasons the UEC launched before having a formal roadmap is that it needs to begin building liaisons with other standards bodies.

So can the UEC really expect to supplant the successful ROCE protocol? Amazon’s SDP provides existence proof that it can be done, at least when one party controls the end-to-end design. The challenge, therefore, is to replicate this effort in an ad-hoc standard with many parties contributing and voting. From a hardware perspective, we expect existing smart NICs can handle the transmit-rate pacing, packet spraying, and out-of-order reception. Multiple Ethernet-switch ASICs already implement packet trimming, also known as drop congestion notification. In our view, the UET must work with at least a subset of existing NICs and switch chips to achieve rapid adoption. The UEC plans to admit new members beginning in 4Q23, the next big milestone in building industry momentum. 

Comments

Post a Comment

Popular posts from this blog

AMD Looks to Infinity for AI Interconnects

Image
With the formal launch of the MI300 GPU, AMD revealed new plans for scaling the multi-GPU interconnects vital to AI-training performance. The company's approach relies on a partner ecosystem, which stands in stark contrast with NVIDIA's end-to-end solutions. The plans revolve around AMD's proprietary Infinity Fabric and its underlying XGMI interconnect. Infinity Fabric Adopts Switching As with its prior generation, AMD uses XGMI to connect multiple MI300 GPUs in what it calls a hive. The hive shares a homogeneous memory space formed by the HBM attached to each GPU. In current designs, the GPUs connect directly using XGMI in a mesh or ring topology. Each MI300X GPU has up to seven Infinity Fabric links, each with 16 lanes. The 4th-gen Infinity Fabric supports up to 32Gbps per lane, yielding 128GB/s of bidirectional bandwidth per link. At the MI300 launch, Broadcom announced that its next-generation PCI Express (PCIe) switch chip will add support for XGMI. At last October'
Read more

NVIDIA Networks NVLink

Image
I attended several sessions at last week's Hot Chips, and NVIDIA's NVSwitch talk was a standout. Ashraf Eassa did a great job of covering the talk's contents in an NVIDIA blog, so I will focus on analysis here.  SuperPOD Bids Adieu to InfiniBand From a system-architecture perspective, the biggest change is extending NVLink beyond a single chassis. NVLink Network is a new protocol built on the NVLink4 link layer. It reuses 400G Ethernet cabling to enable passive-copper (DAC), active-copper (AEC), and optical links. The build its DGX H100 SuperPOD, NVIDIA designed a 1U switch system around a pair of NVSwitch chips. The pod ("scalable unit") includes a central rack with 18 NVLink Switch systems, connecting 32 DGX H100 nodes in a two-level fat-tree topology. This pod interconnect yields 460.8Tbps of bisectional bandwidth. NVLink Network replaces InfiniBand (IB) as the first level of interconnect in DGX SuperPODs. The A100 generation pod uses an IB HDR leaf/spine inte
Read more

NVIDIA Reveals DGX GH200 System Architecture

Image
We industry analysts sometimes get out over our skis when trying to project details of new products. Following NVIDIA's DGX GH200 announcement at Computex, we noticed industry press making the same mistake. Rather than correct our previous NVLink Network post, we'll explain here what we've since learned. Our revelation came when NVIDIA published a white paper titled NVIDIA Grace Hopper Superchip Architecture . The figure below from that paper shows the interconnects at the HGX-module level. The GPU (Hopper) side uses NVLink as a coherent interconnect, whereas the CPU (Grace) side uses InfiniBand, in this case connected through a Bluefield-3 DPU. From a networking perspective, the NVLink and InfiniBand domains are independent, that is, there is no bridging between the two protocols. HGX Grace Hopper Superchip System With NVLink Switch (Source: NVIDIA) The new DGX GH200 builds a SuperPOD based on this underlying module-level architecture. You've probably seen the headlin
Read more