Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Turning the CPU-GPU Hybrid System on Its Head

nextplatform.com

Post reply on HN
Print view View original

11–20 of 32 posts

Re: Turning the CPU-GPU Hybrid System on Its Head

#11

Post by marmaduke »

m_mueller wrote: post #3

There's something qualitatively new in the DGX machines. GPUs have been great for numerics in HPC, but most workloads that matter in HPC (and where there is a market for expensive chips) don't fit on single GPU or even single GPU nodes - until the DGX(2). If they can scale this up (i.e. a tree of NVswitches connecting multiple DGX) it means that suddenly you can treat a whole datacenter worth of hardware as single GP…

The DGX is impressive if incremental progress. MPI, OpenMP and SIMD are different APIs where CUDA is one, but their problems still need solving, eg inter-node, intra-node/inter-core, and intra-core/lane. It still takes a domain expert to map a problem into those levels effectively. CUDA automates this for tree of GPUs but only do this in a handful of relatively straightforward problems.

Re: Turning the CPU-GPU Hybrid System on Its Head

#12

Post by shaklee3 »

m_mueller wrote: post #3

There's something qualitatively new in the DGX machines. GPUs have been great for numerics in HPC, but most workloads that matter in HPC (and where there is a market for expensive chips) don't fit on single GPU or even single GPU nodes - until the DGX(2). If they can scale this up (i.e. a tree of NVswitches connecting multiple DGX) it means that suddenly you can treat a whole datacenter worth of hardware as single GP…

The most impressive part about the dgx2 is the nvswitch architecture. Nvlink was a stepping stone to get there, and nvswitch is the next progression to make a unified address space on up to 16 GPUs across 2 boxes. I think the biggest threat for Nvidia competitors will be switch manufacturers. The biggest bottleneck continues to be the interconnects between cards and nodes. I wouldn't be surprised if Nvidia designs a proprietary external switch that can connect many dgx-2 as the next step, reducing the need for Ethernet or infiniband. I think the impact of nvswitch cannot be understated, and I don't think it gets much press.

Re: Turning the CPU-GPU Hybrid System on Its Head

#13

Post by shaklee3 »

twtw wrote: post #7
baybal2 wrote: post #2

Good for Mr. Huang. It is surprising to see them grow that much in that sector, despite it being quite marginal. GPU's use in corporate sector and HPC will still be dwarfed many times over by sales to gaming PC market

> many times over Where "many times" ~= 2.2, based on the revenue data in the article.

And gaming is flattening, while enterprise isn't.

Re: Turning the CPU-GPU Hybrid System on Its Head

#15

Post by eikenberry »

CodeArtisan wrote: post #10

The IBM Cell, an architecture decried by many at the time, especially in the game industry (Gabe Newell's infamous rant), was right: A lot of specialized cores managed by a few generic, all-purpose, cores. http://www.blachford.info/computer/Cell/Cell0_v2.html

I thought the big problem with the Cell architecture was that fully taking advantage of it was very difficult. Particularly as a gaming system where game engines have to be finely tuned to the hardware. Has this changed?

Re: Turning the CPU-GPU Hybrid System on Its Head

#16

Post by jandrese »

eikenberry wrote: post #15
CodeArtisan wrote: post #10

The IBM Cell, an architecture decried by many at the time, especially in the game industry (Gabe Newell's infamous rant), was right: A lot of specialized cores managed by a few generic, all-purpose, cores. http://www.blachford.info/computer/Cell/Cell0_v2.html

I thought the big problem with the Cell architecture was that fully taking advantage of it was very difficult. Particularly as a gaming system where game engines have to be finely tuned to the hardware. Has this changed?

The big problem with Cell is that it didn't come with a graphics stack. Sony apparently thought developers would be OK with building their own geometry/T&L/etc... engines and then were kind of surprised when that idea turned out to be unpopular. They were then forced to bolt a traditional GPU on the side to appease the developers.

Even if someone was crazy enough to build the entire graphics stack on the Cell, performance would likely not have been as good as a regular graphics card. It's hard to compete with hardware designed from the ground up for the task.

Re: Turning the CPU-GPU Hybrid System on Its Head

#17

Post by arnon »

shaklee3 wrote: post #12
m_mueller wrote: post #3

There's something qualitatively new in the DGX machines. GPUs have been great for numerics in HPC, but most workloads that matter in HPC (and where there is a market for expensive chips) don't fit on single GPU or even single GPU nodes - until the DGX(2). If they can scale this up (i.e. a tree of NVswitches connecting multiple DGX) it means that suddenly you can treat a whole datacenter worth of hardware as single GP…

The most impressive part about the dgx2 is the nvswitch architecture. Nvlink was a stepping stone to get there, and nvswitch is the next progression to make a unified address space on up to 16 GPUs across 2 boxes. I think the biggest threat for Nvidia competitors will be switch manufacturers. The biggest bottleneck continues to be the interconnects between cards and nodes. I wouldn't be surprised if Nvidia designs a…

NVLink between CPU and GPU is the key, and only IBM has implemented that with Power8 and Power9.

Everything NVSwitch does is between GPUs, and doesn't really help with getting data up to the GPUs.

Re: Turning the CPU-GPU Hybrid System on Its Head

#18

Post by m_mueller »

shaklee3 wrote: post #12
m_mueller wrote: post #3

There's something qualitatively new in the DGX machines. GPUs have been great for numerics in HPC, but most workloads that matter in HPC (and where there is a market for expensive chips) don't fit on single GPU or even single GPU nodes - until the DGX(2). If they can scale this up (i.e. a tree of NVswitches connecting multiple DGX) it means that suddenly you can treat a whole datacenter worth of hardware as single GP…

The most impressive part about the dgx2 is the nvswitch architecture. Nvlink was a stepping stone to get there, and nvswitch is the next progression to make a unified address space on up to 16 GPUs across 2 boxes. I think the biggest threat for Nvidia competitors will be switch manufacturers. The biggest bottleneck continues to be the interconnects between cards and nodes. I wouldn't be surprised if Nvidia designs a…

Exactly, that was my point with the tree of NVswitches. That's the obvious next progression. But to have an interconnect that fast, as I wrote, is also a change in mindset in how you can treat that system. A unified address space is no fun if only a small fraction of the memory is available with high throughput on a specific card, so you are essentially back to a version of cache locality optimizations that are really painful to program for in anything that's not a simple parallel data layout. Having that bandwidth to the whole DGX allows entirely new use usecases.

Re: Turning the CPU-GPU Hybrid System on Its Head

#19

Post by shaklee3 »

arnon wrote: post #17
shaklee3 wrote: post #12

Earlier quoted context omitted.

The most impressive part about the dgx2 is the nvswitch architecture. Nvlink was a stepping stone to get there, and nvswitch is the next progression to make a unified address space on up to 16 GPUs across 2 boxes. I think the biggest threat for Nvidia competitors will be switch manufacturers. The biggest bottleneck continues to be the interconnects between cards and nodes. I wouldn't be surprised if Nvidia designs a…

NVLink between CPU and GPU is the key, and only IBM has implemented that with Power8 and Power9. Everything NVSwitch does is between GPUs, and doesn't really help with getting data up to the GPUs.

Going to the CPU is largely irrelevant in most cases. A lot of algorithms just need to transfer data between GPUs or nodes, and RDMA bypasses the CPU completely. NvLink is nice on POWER9, but it's only one or two links instead of the full 6 a GPU has. What we really need is GPUs and CPUs to support PCIe 4/5. Intel's Cascade Lake Xeons don't even support 4, and it's out out until next year. Power has had it for over a year.

Re: Turning the CPU-GPU Hybrid System on Its Head

#20

Post by shaklee3 »

m_mueller wrote: post #18
shaklee3 wrote: post #12

Earlier quoted context omitted.

The most impressive part about the dgx2 is the nvswitch architecture. Nvlink was a stepping stone to get there, and nvswitch is the next progression to make a unified address space on up to 16 GPUs across 2 boxes. I think the biggest threat for Nvidia competitors will be switch manufacturers. The biggest bottleneck continues to be the interconnects between cards and nodes. I wouldn't be surprised if Nvidia designs a…

Exactly, that was my point with the tree of NVswitches. That's the obvious next progression. But to have an interconnect that fast, as I wrote, is also a change in mindset in how you can treat that system. A unified address space is no fun if only a small fraction of the memory is available with high throughput on a specific card, so you are essentially back to a version of cache locality optimizations that are reall…

Yup. There was a great talk at the last GPU tech conference in san jose where a guy from nvidia showed benchmarks of local memory accesses compared to remote (another GPU over nvlink/nvswitch). There was almost no difference once you got to a large enough transfer size, which is the most common. And the beauty was the only difference is the pointer happened to be on another GPU rather than yours. No special programming was required. Of course, this locks you into Nvidia, but their intent is clear with CUDA already.
Post reply on HN