AI Unsurprisingly Dominates Hot Chips 2024
![Image](https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj-fx2yqioQYNVfSWEUuUyGkreqKugQmyJzMzxr6dtjQtt1zlj9yquWmRDLeV95Y4WW-2sPv_VC2enMKdmJx-X1rmU2rnxk7pAJ0mrRn3euFUf7KUWFMPsFWDcIS9iXDg18bMDOPX7IQuOAYv7i6oFEk5z_gieiLy-R5D22f8qc83KJuQq4KbkOr3TcxAU/w438-h640/NVL36%20rack.png)
This year's edition of the annual Hot Chips conference represented the peak in the generative-AI hype cycle. Consistent with the theme, OpenAI's Trevor Cai made the bull case for AI compute in his keynote. At a conference known for technical disclosures, however, the presentations from merchant chip vendors were disappointing; despite a great lineup of talks, few new details emerged. Nvidia's Blackwell presentation mostly rehashed previously disclosed information. In a picture-is-worth-a-thousand-words moment, however, one slide included the photo of the GB200 NVL36 rack shown below. GB200 NVL36 rack (Source: Nvidia) Many customers prefer the NVL36 over the power-hungry NVL72 configuration, which requires a massive 120kW per rack. The key difference for our readers is that the NVLink switch trays shown in the middle of the rack have front-panel cages, whereas the "non-scalable" NVLink switch tray used in the NVL72 has only back-panel connectors for the NVLink spin