AMD EPYC Rome 2P Will Have 128-160 PCIe Gen4 Lanes and a Bonus
servethehome.com
AMD EPYC Rome 2P Will Have 128-160 PCIe Gen4 Lanes and a Bonus
1–10 of 66 posts
Re: AMD EPYC Rome 2P Will Have 128-160 PCIe Gen4 Lanes and a Bonus
#2Can someone enlighten me with some information about application requiring the full 16x PCIe gen4 bandwidth per slot (or 32x gen3 for that matter)? I can imagine some HPC GPU solutions but other than that what requires the thoughput?
There is obviously a market for this as both giants are building platforms.
Re: AMD EPYC Rome 2P Will Have 128-160 PCIe Gen4 Lanes and a Bonus
#3Can someone enlighten me with some information about application requiring the full 16x PCIe gen4 bandwidth per slot (or 32x gen3 for that matter)? I can imagine some HPC GPU solutions but other than that what requires the thoughput? There is obviously a market for this as both giants are building platforms.
High performance networking on the whole, of which HPC GPU solutions are only one use case. And there's the continuing trend of sticking offload engines in the form of ARM ASICs alongside devices that connect over PCIe.
A bit farther into the future: https://genzconsortium.org The idea of everything being connected by a shared high speed fabric enables ideas like shared memory pools across a fabric for 'composable' configurations. https://drivescale.com/software-composable-infrastructure/
Re: AMD EPYC Rome 2P Will Have 128-160 PCIe Gen4 Lanes and a Bonus
#4Can someone enlighten me with some information about application requiring the full 16x PCIe gen4 bandwidth per slot (or 32x gen3 for that matter)? I can imagine some HPC GPU solutions but other than that what requires the thoughput? There is obviously a market for this as both giants are building platforms.
Not an expert, but at least on the gaming side of GPUs my understanding is that you usually aren't using the bandwidth much. For rendering your average frame, all the data you need is already in VRAM.
But if there's new model or textures that aren't yet on the GPU, you want to move them over quickly because the next frame needs to go out ASAP. For a game running at 60hz you've got 16ms to avoid a frame delay. At 144hz you have 7ms. So it can be helpful to be able to dump stuff to VRAM very quickly, which the GPU can grab from system memory via DMA at very high speeds.
For most model/texture loading you know an object is coming into the scene and would have put it on the GPU ahead of time, so this a sort of worst case scenario shouldn't really come up unless you have a huge number of possible things that could spawn in and you don't know which until the last minute. Pretty unlikely.
For the biggest transfers, like loading in an entire level worth of assets, you can hide it with loading screens, or load low resolution textures to start with and stream in the high resolution versions to catch up. Having more PCIe bandwidth will help speed that up if it's in system RAM, but if you're loading it all from disk that's going to be your limiting factor.
I saw some benchmarks years ago where running a high end GPU at x8 or even x4 had pretty minimal impact. With high res textures coming in for 4K now I'd be curious to see someone run that again.
Realistically, I don't think gamers should care about it beyond the spec one-upmanship. HPC would get a lot more use out of it. Maybe also a shift toward fast PCIe based storage solutions like Intel's Optane?
Re: AMD EPYC Rome 2P Will Have 128-160 PCIe Gen4 Lanes and a Bonus
#5Can someone enlighten me with some information about application requiring the full 16x PCIe gen4 bandwidth per slot (or 32x gen3 for that matter)? I can imagine some HPC GPU solutions but other than that what requires the thoughput? There is obviously a market for this as both giants are building platforms.
It is not just about throughput, it is about the flexibility of how you can use the lanes.
Working with desktop computers I've often been frustrated by the limitations of PCIe lanes on Intel processors which are exacerbated by limitations of the motherboards. Frequently I find that "I can't get from here to there" when I am adding cards because the machine has an i5 processor or that particular motherboard, or... Often I can turn off some features on the motherboard i don't need to free up some lanes or shuffle the cards around, but it reminds me of the bad old days when I had to set jumpers for the interrupts on all the cards.
There are numerous holes in the hardware industry that have been caused by Intel's stinginess on I/O. You don't feel the pain on laptops or the tablets and cell phones that Intel had vainglorious ambitions for, but it's been one less reason for people to build machines for which Intel could wipe the floor of their competitors.
It's also been a big aid to IBM's Power architecture because one of the few things keeping that alive is better I/O.
Re: AMD EPYC Rome 2P Will Have 128-160 PCIe Gen4 Lanes and a Bonus
#6Can someone enlighten me with some information about application requiring the full 16x PCIe gen4 bandwidth per slot (or 32x gen3 for that matter)? I can imagine some HPC GPU solutions but other than that what requires the thoughput? There is obviously a market for this as both giants are building platforms.
A good target is storage, ie a lot of NVMe drives.
Re: AMD EPYC Rome 2P Will Have 128-160 PCIe Gen4 Lanes and a Bonus
#7Can someone enlighten me with some information about application requiring the full 16x PCIe gen4 bandwidth per slot (or 32x gen3 for that matter)? I can imagine some HPC GPU solutions but other than that what requires the thoughput? There is obviously a market for this as both giants are building platforms.
A good target is storage, ie a lot of NVMe drives.
Yeah, that's why most PCIe M2 top out at around 3500 MB/s, I doubt it will take long to saturate 7 GB/s (or double that in RAID0). Either way, it's a heck of a lot of bandwidth!
Re: AMD EPYC Rome 2P Will Have 128-160 PCIe Gen4 Lanes and a Bonus
#8Can someone enlighten me with some information about application requiring the full 16x PCIe gen4 bandwidth per slot (or 32x gen3 for that matter)? I can imagine some HPC GPU solutions but other than that what requires the thoughput? There is obviously a market for this as both giants are building platforms.
Nvme and 8k displays?
Re: AMD EPYC Rome 2P Will Have 128-160 PCIe Gen4 Lanes and a Bonus
#9Can someone enlighten me with some information about application requiring the full 16x PCIe gen4 bandwidth per slot (or 32x gen3 for that matter)? I can imagine some HPC GPU solutions but other than that what requires the thoughput? There is obviously a market for this as both giants are building platforms.
Virtualization does, most virtualization platforms require dedicated lanes for each VM and devices assigned to the VM for dedicated situations.
Re: AMD EPYC Rome 2P Will Have 128-160 PCIe Gen4 Lanes and a Bonus
#10Can someone enlighten me with some information about application requiring the full 16x PCIe gen4 bandwidth per slot (or 32x gen3 for that matter)? I can imagine some HPC GPU solutions but other than that what requires the thoughput? There is obviously a market for this as both giants are building platforms.
NVMeOF (NVMe Over Fabrics) comes to mind, having multiple NVMe drives (mostly PCIe x4) + networking (100GbE uses x16 PCIe) consumes a lot of PCIe lanes.
Also note that current two-socket AMD Epyc servers use 64 lanes from each CPU for inter-CPU communication, so the entire server has only 128 lanes (out of 256) left for peripherals.