Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD EPYC Rome 2P Will Have 128-160 PCIe Gen4 Lanes and a Bonus

servethehome.com

Post reply on HN
Print view View original

11–20 of 66 posts

Re: AMD EPYC Rome 2P Will Have 128-160 PCIe Gen4 Lanes and a Bonus

#11

Post by jamesblonde »

consp wrote: post #2

Can someone enlighten me with some information about application requiring the full 16x PCIe gen4 bandwidth per slot (or 32x gen3 for that matter)? I can imagine some HPC GPU solutions but other than that what requires the thoughput? There is obviously a market for this as both giants are building platforms.

This will give us servers for deep learning that can have 8 GPUs and a couple of NVMe disks on PCI 4.0 (32 GB/s). With very good inter-GPU I/O and access to NVMe, it will enable commodity servers that are competitive with Nvidia's DGX-1 or DGX2, that include SXM2 (Nvlink with 80GB/s between GPUs).

Re: AMD EPYC Rome 2P Will Have 128-160 PCIe Gen4 Lanes and a Bonus

#12

Post by masklinn »

consp wrote: post #2

Can someone enlighten me with some information about application requiring the full 16x PCIe gen4 bandwidth per slot (or 32x gen3 for that matter)? I can imagine some HPC GPU solutions but other than that what requires the thoughput? There is obviously a market for this as both giants are building platforms.

The 16x links will usually get split out to 4x nvme slots, and that's where you're going to get limited by the PCIe bandwidth.

Re: AMD EPYC Rome 2P Will Have 128-160 PCIe Gen4 Lanes and a Bonus

#13

Post by dodobirdlord »

consp wrote: post #2

Can someone enlighten me with some information about application requiring the full 16x PCIe gen4 bandwidth per slot (or 32x gen3 for that matter)? I can imagine some HPC GPU solutions but other than that what requires the thoughput? There is obviously a market for this as both giants are building platforms.

GPU training of ML/DL models is bottlenecked in bizarre locations. Generally when doing distributed training on multiple GPUs with a large dataset you move all at once enough data to fill the GPU RAM during training and let it crunch for a little while, then push the next batch in. People have come up with workarounds like doing multiple training passes over each batch of data, but ideally you would not do that and would refresh the whole 12/16GB of training data on each pass. If you have enough RAM on the board to keep the whole training dataset in memory (likely) then you can easily find that the bottleneck in your system is bandwidth to the GPU. People like to be able to train with a large number of GPUs in parallel, but they really don't like cutting down to 8x PCIe lanes/GPU.

Re: AMD EPYC Rome 2P Will Have 128-160 PCIe Gen4 Lanes and a Bonus

#14

Post by kilo_bravo_3 »

consp wrote: post #2

Can someone enlighten me with some information about application requiring the full 16x PCIe gen4 bandwidth per slot (or 32x gen3 for that matter)? I can imagine some HPC GPU solutions but other than that what requires the thoughput? There is obviously a market for this as both giants are building platforms.

Off the top of my head this would be great for massive enterprise VDI installations and game streaming services (which is just VDI by another name), ML, AI applications, and NVMe storage.

Personally, this will be good for me because everything I do is bandwidth-starved.

I'm not a radar scientist but I am a systems engineer supporting radar scientists working on air- and space-based Synthetic Aperture Radar (SAR) systems. We use GPUs, FPGAs, and other accelerators to generate images from SAR data.

Here's an "old and busted" image made from SAR data: https://hackadaycom.files.wordpress.com/2014/02/image-from-s...

In 2002, it took more than 24 hours to generate a single low-resolution picture from low-bandwidth SAR data on a $1.5 million Sunfire 15K cluster with 70-ish SPARC CPUs. Today, on a single 3U server with two Xeons and four Tesla V100s it takes about 15 seconds-- and that's an extremely high resolution image from very high data rate SAR data.

But our goal is real-time VIDEO from SAR data, so everything needs to be faster. Network speeds need to be faster, CPUs need to be faster, GPUs need to be faster and we need more of them, storage needs to be faster, everything needs to be faster.

I could see a 3/4U box with 16+ 16x PCIe slots each stuffed with a 1-slot GPU with an NVMe (4x PCIe lanes each!) storage array and a couple of 100GBe dual-port NICs blasting through SAR data like a hungry hungry hippo.

As far as PCIe lanes go, if I have a 24-drive NVMe array that's 96 PCIe lanes all by itself.

Re: AMD EPYC Rome 2P Will Have 128-160 PCIe Gen4 Lanes and a Bonus

#15

Post by consp »

kilo_bravo_3 wrote: post #14
consp wrote: post #2

Can someone enlighten me with some information about application requiring the full 16x PCIe gen4 bandwidth per slot (or 32x gen3 for that matter)? I can imagine some HPC GPU solutions but other than that what requires the thoughput? There is obviously a market for this as both giants are building platforms.

Off the top of my head this would be great for massive enterprise VDI installations and game streaming services (which is just VDI by another name), ML, AI applications, and NVMe storage. Personally, this will be good for me because everything I do is bandwidth-starved. I'm not a radar scientist but I am a systems engineer supporting radar scientists working on air- and space-based Synthetic Aperture Radar (SAR) syst…

That makes sense. Thanks to the reactions here (from you and others) getting an idea of the bandwidth starved computational tasks in real life.

Re: AMD EPYC Rome 2P Will Have 128-160 PCIe Gen4 Lanes and a Bonus

#17

Post by justinclift »

kilo_bravo_3 wrote: post #14
consp wrote: post #2

Can someone enlighten me with some information about application requiring the full 16x PCIe gen4 bandwidth per slot (or 32x gen3 for that matter)? I can imagine some HPC GPU solutions but other than that what requires the thoughput? There is obviously a market for this as both giants are building platforms.

Off the top of my head this would be great for massive enterprise VDI installations and game streaming services (which is just VDI by another name), ML, AI applications, and NVMe storage. Personally, this will be good for me because everything I do is bandwidth-starved. I'm not a radar scientist but I am a systems engineer supporting radar scientists working on air- and space-based Synthetic Aperture Radar (SAR) syst…

Hmmm, how many radar pulses need to hit a target to generate video? Sounds potentially irradating... ;)

eg: WARNING Don't attempt to image living creatures WARNING

Re: AMD EPYC Rome 2P Will Have 128-160 PCIe Gen4 Lanes and a Bonus

#18

Post by dragontamer »

consp wrote: post #2

Can someone enlighten me with some information about application requiring the full 16x PCIe gen4 bandwidth per slot (or 32x gen3 for that matter)? I can imagine some HPC GPU solutions but other than that what requires the thoughput? There is obviously a market for this as both giants are building platforms.

Virtualization.

Remember that these EPYC systems will have up to 64 cores per chip. 128 cores / 256 threads per 2P platform, with 128x to 160x PCIe 4.0 lanes depending on the motherboard.

Lets say you split up your system into 32 VMs (4-cores / 8-threads per VM). That's only 4x to 6x PCIe 4.0 lanes per VM. Doesn't seem very spacious anymore, does it? That's barely enough room for a NVMe drive per VM + a small, shared chunk of graphics (modern enterprise GPUs can be virtualized and split between VMs. So one x16 link to a beefy GPU can be shared between 8 VMs)

If your current customer is using NVMe drives + thick-client workstations for docker / testing / etc. etc, it may be difficult to convince them to switch to a virtualized infrastructure. But a beefy machine with this much I/O would definitely catch their eyes.

Re: AMD EPYC Rome 2P Will Have 128-160 PCIe Gen4 Lanes and a Bonus

#19

Post by en4bz »

consp wrote: post #2

Can someone enlighten me with some information about application requiring the full 16x PCIe gen4 bandwidth per slot (or 32x gen3 for that matter)? I can imagine some HPC GPU solutions but other than that what requires the thoughput? There is obviously a market for this as both giants are building platforms.

Dual port 100 and 200 Gbit/s network adapters.
Post reply on HN