Intel Gaudi 3 AI Accelerator • Hacker News Board

Re: Intel Gaudi 3 AI Accelerator

#131

Post by KeplerBoy » Tue, Apr 09, 2024, 9:02 PM UTC

formerly_proven wrote: post #130

mk_stjames wrote: post #111

One nice thing about this (and the new offerings from AMD) is that they will be using the "open accelerator module (OAM)" interface- which standardizes the connector that they use to put them on baseboards, similar to the SXM connections of Nvidia that use MegArray connectors to thier baseboards. With Nvidia, the SXM connection pinouts have always been held proprietary and confidential. For example, P100's and V100's…

The price is low because they’re useless (except for replacing dead cards in a DGX), if you had a 40$ PCIe AIC-to-SXM adapter, the price would go up a lot. > I'm one of those people who finds 'retro-super-computing' a cool hobby and thus the interfaces like OAM being open means that these devices may actually have a life for hobbyists in 8~10 years instead of being sent directly to the bins due to secret interfaces a…

I'm not sure the prices would go up that much. What would anyone buy that card for?

Yes, it has a decent memory bandwidth (~750 GB/s) and it runs CUDA. But it only has 16 GB and doesn't support tensor cores or low precision floats. It's in a weird place.

Re: Intel Gaudi 3 AI Accelerator

#132

Post by meragrin_ » Tue, Apr 09, 2024, 9:05 PM UTC

geertj wrote: post #103

I wonder if someone knowledgeable could comment on OneAPI vs Cuda. I feel like if Intel is going to be a serious competitor to Nvidia, both software and hardware are going to be equally important.

Apparently, Google, Qualcomm, Samsung, and ARM are rallying around oneAPI:

https://uxlfoundation.org/

Re: Intel Gaudi 3 AI Accelerator

#133

Post by KeplerBoy » Tue, Apr 09, 2024, 9:06 PM UTC

alecco wrote: post #57

Gaudi 3 has PCIe 4.0 (vs. H100 PCIe 5.0, so 2x the bandwidth). Probably not a deal-breaker but it's strange for Intel (of all vendors) to lag behind in PCIe.

The whitepaper says it's PCIe 5 on Gaudi 3.

Re: Intel Gaudi 3 AI Accelerator

#134

Post by bayindirh » Tue, Apr 09, 2024, 9:07 PM UTC

astrodust wrote: post #117

chucke1992 wrote: post #98

Earlier quoted context omitted.

Q6600 was quite good but E8400 was the best.

Q6600 is the spiritual successor to the ABIT BP6 Dual Celeron option: https://en.wikipedia.org/wiki/ABIT_BP6

ABIT was a legend in motherboards. I used their AN-7 Ultra and AN-8 Ultra. No newer board gave the flexibility and capabilities of these series.

My latest ASUS was good enough, but I didn't (and probably won't) build any newer systems, so ABITs will have the crown.

Re: Intel Gaudi 3 AI Accelerator

#135

Post by trueismywork » Tue, Apr 09, 2024, 9:08 PM UTC

KeplerBoy wrote: post #131

formerly_proven wrote: post #130

Earlier quoted context omitted.

The price is low because they’re useless (except for replacing dead cards in a DGX), if you had a 40$ PCIe AIC-to-SXM adapter, the price would go up a lot. > I'm one of those people who finds 'retro-super-computing' a cool hobby and thus the interfaces like OAM being open means that these devices may actually have a life for hobbyists in 8~10 years instead of being sent directly to the bins due to secret interfaces a…

I'm not sure the prices would go up that much. What would anyone buy that card for? Yes, it has a decent memory bandwidth (~750 GB/s) and it runs CUDA. But it only has 16 GB and doesn't support tensor cores or low precision floats. It's in a weird place.

Scientific computing would buy it up like hot cakes.

Re: Intel Gaudi 3 AI Accelerator

#136

Post by amelius » Tue, Apr 09, 2024, 9:08 PM UTC

Missing in these pictures are the thermal management solutions.

Re: Intel Gaudi 3 AI Accelerator

#137

Post by KeplerBoy » Tue, Apr 09, 2024, 9:13 PM UTC

vector floating point performance comes in at 14 Tflops/s for FP32 and 28 Tflop/s for FP16.

Not the best of times for stuff that doesn't fit matrix processing units.

Re: Intel Gaudi 3 AI Accelerator

#138

Post by buildbot » Tue, Apr 09, 2024, 9:14 PM UTC

mk_stjames wrote: post #111

One nice thing about this (and the new offerings from AMD) is that they will be using the "open accelerator module (OAM)" interface- which standardizes the connector that they use to put them on baseboards, similar to the SXM connections of Nvidia that use MegArray connectors to thier baseboards. With Nvidia, the SXM connection pinouts have always been held proprietary and confidential. For example, P100's and V100's…

The SXM2 interface is actually publicly documented! There is an open compute spec for a 8-way baseboard. You can find the pinouts there.

Re: Intel Gaudi 3 AI Accelerator

#139

Post by KeplerBoy » Tue, Apr 09, 2024, 9:18 PM UTC

trueismywork wrote: post #135

KeplerBoy wrote: post #131

Earlier quoted context omitted.

I'm not sure the prices would go up that much. What would anyone buy that card for? Yes, it has a decent memory bandwidth (~750 GB/s) and it runs CUDA. But it only has 16 GB and doesn't support tensor cores or low precision floats. It's in a weird place.

Scientific computing would buy it up like hot cakes.

Only if the specific workload needs FP64 (4.5 Tflop/s), the 9 Tflop/s for FP32 can be had for cheap with Turing or Ampere consumer cards.

Still, your point stands. It's crazy how that 2016 GPU has two thirds the FP32 power of this new 2024 unobtanium card and infinitely more FP64.

Re: Intel Gaudi 3 AI Accelerator

#140

Post by einpoklum » Tue, Apr 09, 2024, 9:20 PM UTC

If your metric is memory bandwidth or memory size, then this announcement gives you some concrete information. But - suppose my metric for performance is matrix-multiply-add (or just matrix-multiply) bandwidth. What MMA primitives does Gaudi offer (i.e. type combinations and matrix dimension combinations), and how many of such ops per second, in practice? The linked page says "64,000 in parallel", but that does not actually tell me much.