Before you read the article: Nvidia stock fell approximately 20% yesterday.
The drop was caused by mismanaged inventories. Mostly related to cryptominign boom cooling down.
Does this mean I'll actually be able to afford a new gaming rig now?
Turning the CPU-GPU Hybrid System on Its Head
21–30 of 32 posts
Re: Turning the CPU-GPU Hybrid System on Its Head
#22Good for Mr. Huang. It is surprising to see them grow that much in that sector, despite it being quite marginal. GPU's use in corporate sector and HPC will still be dwarfed many times over by sales to gaming PC market
> many times over Where "many times" ~= 2.2, based on the revenue data in the article.
I checked it out, and indeed. It went up from 10% a year ago to a fifth of the revenue. Impressive, but it is still an itching question how much of this is due to an AI/crypto bubble and a few big supercomputers coming online in coming years.
Re: Turning the CPU-GPU Hybrid System on Its Head
#23Re: Turning the CPU-GPU Hybrid System on Its Head
#24Earlier quoted context omitted.
NVLink between CPU and GPU is the key, and only IBM has implemented that with Power8 and Power9. Everything NVSwitch does is between GPUs, and doesn't really help with getting data up to the GPUs.
Going to the CPU is largely irrelevant in most cases. A lot of algorithms just need to transfer data between GPUs or nodes, and RDMA bypasses the CPU completely. NvLink is nice on POWER9, but it's only one or two links instead of the full 6 a GPU has. What we really need is GPUs and CPUs to support PCIe 4/5. Intel's Cascade Lake Xeons don't even support 4, and it's out out until next year. Power has had it for over a…
RDMA may bypass the CPU but it doesn't bypass PCIe. It still goes through it, which is slower than NVLink on Power9 (~16GB/s on 16x PCIe 3.0 vs ~150GB/s on NVLink).
Data has to be copied up to the GPU for processing. It has to be brought down to be sent as a result set somewhere. If everything you're doing fits in GPU memory, you've only done that once. If it doesn't, you need to copy data up and down. You may need to do that for intermediate results too.
This adds up.
Re: Turning the CPU-GPU Hybrid System on Its Head
#25Earlier quoted context omitted.
I thought the big problem with the Cell architecture was that fully taking advantage of it was very difficult. Particularly as a gaming system where game engines have to be finely tuned to the hardware. Has this changed?
The big problem with Cell is that it didn't come with a graphics stack. Sony apparently thought developers would be OK with building their own geometry/T&L/etc... engines and then were kind of surprised when that idea turned out to be unpopular. They were then forced to bolt a traditional GPU on the side to appease the developers. Even if someone was crazy enough to build the entire graphics stack on the Cell, perfor…
Software support (I took an IBM workshop on them) was atrocious. If they had a way to share parts of the SPU memory with a small block shared between adjacent units, it'd be much easier to build a GPU-like thing out of them. The way they were delivered, it was just torture.
Re: Turning the CPU-GPU Hybrid System on Its Head
#26Earlier quoted context omitted.
Going to the CPU is largely irrelevant in most cases. A lot of algorithms just need to transfer data between GPUs or nodes, and RDMA bypasses the CPU completely. NvLink is nice on POWER9, but it's only one or two links instead of the full 6 a GPU has. What we really need is GPUs and CPUs to support PCIe 4/5. Intel's Cascade Lake Xeons don't even support 4, and it's out out until next year. Power has had it for over a…
RDMA may bypass the CPU but it doesn't bypass PCIe. It still goes through it, which is slower than NVLink on Power9 (~16GB/s on 16x PCIe 3.0 vs ~150GB/s on NVLink). Data has to be copied up to the GPU for processing. It has to be brought down to be sent as a result set somewhere. If everything you're doing fits in GPU memory, you've only done that once. If it doesn't, you need to copy data up and down. You may need t…
It doesn't matter if it bypasses pcie. The fastest available ethernet interface today is 100Gbps. Pcie gen3 x16 will get you to about 115Gbps, so the limiter is the interface speed. Pcie4 should be common when 200G interfaces come out. This is why an external nvswitch can make network switches vulnerable in this market.
Power 9 has pcie4, so you could theoretically already double your pcie bandwidth with a mellanox card that supports it, but from my previous point, it doesn't matter. Those nvlink numbers are disingenuous. That's the total links to/from a GPU. All available power 9 machines split those links between the other GPUs and CPUs, so you really only have up to 3 of those in the best case, or 75GBps. That's only a bit more than double pcie 4 at that point. And also, by using those nvlinks back to the processor, you are making a decision that you don't need as much bandwidth between GPUs. With the nvswitch architecture, you get the best of both worlds with a full, non-blocking switch.
Re: Turning the CPU-GPU Hybrid System on Its Head
#27Earlier quoted context omitted.
Going to the CPU is largely irrelevant in most cases. A lot of algorithms just need to transfer data between GPUs or nodes, and RDMA bypasses the CPU completely. NvLink is nice on POWER9, but it's only one or two links instead of the full 6 a GPU has. What we really need is GPUs and CPUs to support PCIe 4/5. Intel's Cascade Lake Xeons don't even support 4, and it's out out until next year. Power has had it for over a…
RDMA may bypass the CPU but it doesn't bypass PCIe. It still goes through it, which is slower than NVLink on Power9 (~16GB/s on 16x PCIe 3.0 vs ~150GB/s on NVLink). Data has to be copied up to the GPU for processing. It has to be brought down to be sent as a result set somewhere. If everything you're doing fits in GPU memory, you've only done that once. If it doesn't, you need to copy data up and down. You may need t…
This is true, but it is essentially the same situation as using the CPU for processing.
Re: Turning the CPU-GPU Hybrid System on Its Head
#28This article makes me wonder if we could see a Tegra successor for the data center: a chip with a few ARM cores that really just exist to coordinate the GPU cores and handle I/O.
Re: Turning the CPU-GPU Hybrid System on Its Head
#29The IBM Cell, an architecture decried by many at the time, especially in the game industry (Gabe Newell's infamous rant), was right: A lot of specialized cores managed by a few generic, all-purpose, cores. http://www.blachford.info/computer/Cell/Cell0_v2.html
I thought the big problem with the Cell architecture was that fully taking advantage of it was very difficult. Particularly as a gaming system where game engines have to be finely tuned to the hardware. Has this changed?
The real problem with Cell is it came after X360. Everyone had written and/or ported their engines for heterogeneous computing architectures with no addressing limits(PS3 Cell had max 256kb per SPU).
If you but your engine for PS3 first and ported to X360 it generally ran faster due to better cache coeherancy. Sadly, few did that.
Source: shipped a few titles from that era.
Re: Turning the CPU-GPU Hybrid System on Its Head
#30Earlier quoted context omitted.
Going to the CPU is largely irrelevant in most cases. A lot of algorithms just need to transfer data between GPUs or nodes, and RDMA bypasses the CPU completely. NvLink is nice on POWER9, but it's only one or two links instead of the full 6 a GPU has. What we really need is GPUs and CPUs to support PCIe 4/5. Intel's Cascade Lake Xeons don't even support 4, and it's out out until next year. Power has had it for over a…
RDMA may bypass the CPU but it doesn't bypass PCIe. It still goes through it, which is slower than NVLink on Power9 (~16GB/s on 16x PCIe 3.0 vs ~150GB/s on NVLink). Data has to be copied up to the GPU for processing. It has to be brought down to be sent as a result set somewhere. If everything you're doing fits in GPU memory, you've only done that once. If it doesn't, you need to copy data up and down. You may need t…
There’s actually six 50 GB/s nvlinks on each Power9 cpu (300GB/s), 2 or 3 to each GPU, depending on if you have a 4 or 6 GPU system. I’ve seen speculation that the DGX-2 nvswitches has enough free ports to fully connect in 2 Power9 cpus with 300 GB/s each, that would be an interesting system..