Summary Bullets:
- On October 10, 2024, AMD announced the latest editions of its DPU lineup from its acquisition of Pensando.
- The UEC 1.0 standard will be a big deal in HPC and AI circles.
One of the drivers of high-end networking on the server side is the data processing unit (DPU) also sometimes referred to somewhat erroneously as NIC accelerators. These are add-in networking and services cards for servers in high-performance or hyperscaler environments. Not only do these cards handle traditional NIC acceleration tasks and encryption offloading, but they have their own CPUs and memory. They can support advanced processing and are often used for security and monitoring, among other specialized things – the cards are programable. These cards run outside of the operating system on the server and most often feature ARM-based processors. By running separately on their own operating system, they are extremely useful in security and isolation from the base platform.
AMD, Nvidia, and Intel all have flavors of these cards. DPU cards can not only add to the observability and security of a given environment, they can perform tasks such as complex encryption, load balancing, stateful firewall, NAT, and storage offload to just name a few. They also save CPU cycles on the systems they are installed in. At scale, this can mean considerable savings – the more processing cycles available on each server means fewer overall servers. So, the advantages of these cards in high-performance computing and in hyperscalers are clear. However, for standard generalized enterprise data centers the costs often outweigh the benefits, scale is required to get benefits on a pure CPU cycle savings basis. DPUs also come embedded in switches, bringing advanced services to the network outside of the server.
On October 10, 2024, AMD announced the latest editions of its DPU lineup from its acquisition of Pensando. These two new chips are referred to as the AMD Pensando Pollara 400 and the AMD Pensando Salina 400. These new products are sampling currently and are scheduled for availability in H1 2025.
The AMD Pensando Salina 400 DPU features 2 400 GbE ports, can feature up to 16x ARM cores and up to 128 Gb of memory. This is designed for HPC environments and embedded applications, where extended services are the goal. AMD claims that this new silicon provides up to 2x the performance of its last generation Pensando DPUs. The AMD Pensando Pollara 400 is designed as an AI NIC for advanced AI clusters. It can provide 400 Gbps of bandwidth with a maximum of four ports, meaning a single 400 GbE port, 2 200 GbE ports, etc. It uses fully programmable RDMA transport as well as hardware-based congestion control. More interestingly, it is listed as Ultra Ethernet Consortium Ready.
The Ultra Ethernet Consortium is a standards body whose mission is to “Deliver an Ethernet based open, interoperable, high performance, full-communications stack architecture to meet the growing network demands of AI & HPC at scale.” Essentially, it is creating standards to make Ethernet the transport in AI and HPC, rather than more niche technologies such as InfiniBand. Members of this consortium include but are not limited to industry heavyweights such as Cisco, AMD, Broadcom, Intel, Meta, Microsoft, Arista, Huawei, IBM, Juniper, Marvell, Nvidia, and HPE. The UEC 1.0 standard is due to be published sometime in Q1 2025. The UEC Ready designation means that the hardware meets the requirements to be listed as ‘ready’ for the standard when it is published. The UEC 1.0 standard will be a big deal in HPC and AI circles. When it comes to networking, Ethernet always wins in the end, and it is not expected to be any different when it comes to the HPC and AI environments of the future.