Preview: Hot Chips Returns to Stanford for HC35

I will be at Stanford at the end of this month for the in-person return of Hot Chips. As always, the 35th edition (HC35) will have plenty of deep technical content, with AI/ML unsurprisingly getting lots of attention. I'm particularly interested in a set of talks exploring interconnects and networking for AI, HPC, and beyond.

Day 2 (Tuesday, August 29) features an ML-Training session with talks from Google and Cerebras. The technical lead for TPUs at Google, Norm Jouppi will expand on the paper presented at ISCA 2023 describing the TPUv4 supercomputer. That paper revealed Google's use of optical circuit switches in its TPUv4 cluster, following prior disclosures around OCS deployments in its data-center spine layer.

Sean Li, cofounder and chief hardware architect at Cerebras, will deliver a talk on the company's cluster architecture built around the CS-2 system and WSE-2 wafer-scale engine. This talk will explore how the MemoryX external-memory system and SwarmX fabric interconnect stream data across up to 192 CS-2 systems.

The Interconnects session, also on Day 2, includes talks from NVIDIA, Lightelligence, and Intel. An NVIDIA software architect, Omer Shabtai, will present a Resource-Fungible Network Processing ASIC, building on a vision presentation at NSDI 2022. The talk will discuss NIC programmability using match/action languages such as P4.

VP of engineering at silicon-photonics-startup Lightelligence, Maurice Steinman will disclose new details of the recently announced Hummingbird AI accelerator. This 3D chip stacks a compute die with 64 SIMD cores on top of a silicon-photonic die that provides an optical network-on-a-chip (NoC).

Moving into the research realm, Joshua Fryman, a Fellow and system architect at Intel, will talk about a mesh-to-mesh photonic fabric developed under the DARPA HIVE program. It will reveal that Intel's PUMA graph-computation ASIC implements 1TB/s copackaged optics to create a glueless low-latency interconnect between ASIC sockets.

There will be many more talks on AI/ML hardware, including the Day 1 keynote from Google and the Day 2 keynote from NVIDIA, but I'm excited that interconnects are back in the spotlight!

Comments

Post a Comment

Popular posts from this blog

AMD Looks to Infinity for AI Interconnects

Image
With the formal launch of the MI300 GPU, AMD revealed new plans for scaling the multi-GPU interconnects vital to AI-training performance. The company's approach relies on a partner ecosystem, which stands in stark contrast with NVIDIA's end-to-end solutions. The plans revolve around AMD's proprietary Infinity Fabric and its underlying XGMI interconnect. Infinity Fabric Adopts Switching As with its prior generation, AMD uses XGMI to connect multiple MI300 GPUs in what it calls a hive. The hive shares a homogeneous memory space formed by the HBM attached to each GPU. In current designs, the GPUs connect directly using XGMI in a mesh or ring topology. Each MI300X GPU has up to seven Infinity Fabric links, each with 16 lanes. The 4th-gen Infinity Fabric supports up to 32Gbps per lane, yielding 128GB/s of bidirectional bandwidth per link. At the MI300 launch, Broadcom announced that its next-generation PCI Express (PCIe) switch chip will add support for XGMI. At last October...
Read more

NVIDIA Networks NVLink

Image
I attended several sessions at last week's Hot Chips, and NVIDIA's NVSwitch talk was a standout. Ashraf Eassa did a great job of covering the talk's contents in an NVIDIA blog, so I will focus on analysis here.  SuperPOD Bids Adieu to InfiniBand From a system-architecture perspective, the biggest change is extending NVLink beyond a single chassis. NVLink Network is a new protocol built on the NVLink4 link layer. It reuses 400G Ethernet cabling to enable passive-copper (DAC), active-copper (AEC), and optical links. The build its DGX H100 SuperPOD, NVIDIA designed a 1U switch system around a pair of NVSwitch chips. The pod ("scalable unit") includes a central rack with 18 NVLink Switch systems, connecting 32 DGX H100 nodes in a two-level fat-tree topology. This pod interconnect yields 460.8Tbps of bisectional bandwidth. NVLink Network replaces InfiniBand (IB) as the first level of interconnect in DGX SuperPODs. The A100 generation pod uses an IB HDR leaf/spine inte...
Read more

Ultra Ethernet Promises New RDMA Protocol

Image
This week saw the formal launch of the Ultra Ethernet Consortium (UEC), which aims to reinvent Ethernet fabrics for massive-scale AI and HPC deployments. An impressive list of founding members back this ambitious effort: hyperscalers Meta and Microsoft; chip vendors AMD, Broadcom, and Intel; OEMs Arista, Atos, and HPE; and Cisco, which straddles the chip and OEM camps. Absent this backing, we could easily write off this consortium as doomed to failure. Our skepticism is rooted not in the obvious need the UEC looks to serve but rather in the challenges of standardizing and implementing a full-stack approach. The effort plans to replace existing transport protocols as well as user-space APIs. Specifically, the Ultra Ethernet Transport (UET) protocol will be a new RDMA protocol to replace ROCE, and new APIs will replace the Verbs API from the InfiniBand heritage. UET will provide an alternative to RoCEv2 and Amazon’s SRD , both of which are deployed in hyperscale data centers. (Source: Ul...
Read more