Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel Gaudi 3 AI Accelerator

intel.com

Post reply on HN
Print view View original

61–70 of 260 posts

Re: Intel Gaudi 3 AI Accelerator

#61

Post by bayindirh »

neilmovva wrote: post #12

A bit surprised that they're using HBM2e, which is what Nvidia A100 (80GB) used back in 2020. But Intel is using 8 stacks here, so Gaudi 3 achieves comparable total bandwidth (3.7TB/s) to H100 (3.4TB/s) which uses 5 stacks of HBM3. Hopefully the older HBM has better supply - HBM3 is hard to get right now! The Gaudi 3 multi-chip package also looks interesting. I see 2 central compute dies, 8 HBM die stacks, and then 6…

> A bit surprised that they're using HBM2e, which is what Nvidia A100 (80GB) used back in 2020.

This is one of the secret recipes of Intel. They can use older tech and push it a little further to catch/surpass current gen tech until current gen becomes easier/cheaper to produce/acquire/integrate.

They have done it with their first quad core processors by merging two dual core processors (Q6xxx series), or by creating absurdly clocked single core processors aimed at very niche market segments.

We have not seen it until now, because they were sleeping at the wheel, and knocked unconscious by AMD.

Re: Intel Gaudi 3 AI Accelerator

#62

Post by forkerenok »

kylixz wrote: post #53

This is a bit snarky — but will Intel actually keep this product line alive for more than a few years? Having been bitten by building products around some of their non-x86 offerings where they killed good IP off and then failed to support it… I’m skeptical. I truly do hope it is successful so we can have some alternative accelerators.

I'm not very involved in the broader topic, but isn't the shortage of hardware for AI-related workloads intense enough so as to grant them the benefit of the doubt?

Re: Intel Gaudi 3 AI Accelerator

#64

Post by SteveNuts »

gnabgib wrote: post #5
1024core wrote: post #2

> Memory Boost for LLM Capacity Requirements: 128 gigabytes (GB) of HBMe2 memory capacity, 3.7 terabytes (TB) of memory bandwidth ... I didn't know "terabytes (TB)" was a unit of memory bandwidth...

Bit of an embarrassing typo, they do later qualify it as 3.7TB/s

Most of the time bandwidth is expressed in giga/gibi/tera/tebi bits per second so this is also confusing to me

Re: Intel Gaudi 3 AI Accelerator

#66

Post by jtriangle »

kylixz wrote: post #53

This is a bit snarky — but will Intel actually keep this product line alive for more than a few years? Having been bitten by building products around some of their non-x86 offerings where they killed good IP off and then failed to support it… I’m skeptical. I truly do hope it is successful so we can have some alternative accelerators.

The real question is, how long does it actually have to hang around really? With the way this market is going, it probably only has to be supported in earnest for a few years by which point it'll be so far obsolete that everyone who matters will have moved on.

Re: Intel Gaudi 3 AI Accelerator

#67

Post by mvkel »

bayindirh wrote: post #61
neilmovva wrote: post #12

A bit surprised that they're using HBM2e, which is what Nvidia A100 (80GB) used back in 2020. But Intel is using 8 stacks here, so Gaudi 3 achieves comparable total bandwidth (3.7TB/s) to H100 (3.4TB/s) which uses 5 stacks of HBM3. Hopefully the older HBM has better supply - HBM3 is hard to get right now! The Gaudi 3 multi-chip package also looks interesting. I see 2 central compute dies, 8 HBM die stacks, and then 6…

> A bit surprised that they're using HBM2e, which is what Nvidia A100 (80GB) used back in 2020. This is one of the secret recipes of Intel. They can use older tech and push it a little further to catch/surpass current gen tech until current gen becomes easier/cheaper to produce/acquire/integrate. They have done it with their first quad core processors by merging two dual core processors (Q6xxx series), or by creating…

Interesting.

Would you say this means Intel is "back," or just not completely dead?

Re: Intel Gaudi 3 AI Accelerator

#68

Post by TheAceOfHearts »

whalesalad wrote: post #31

https://www.merriam-webster.com/dictionary/gaudy

Honestly, I thought the same thing upon reading the name. I'm aware of the reference to Antoni Gaudí, but having the name sound so close to gaudy seems a bit unfortunate. Surely they must've had better options? Then again I don't know how these sorts of names get decided anymore.

Re: Intel Gaudi 3 AI Accelerator

#69

Post by alecco »

wmf wrote: post #60
alecco wrote: post #57

Gaudi 3 has PCIe 4.0 (vs. H100 PCIe 5.0, so 2x the bandwidth). Probably not a deal-breaker but it's strange for Intel (of all vendors) to lag behind in PCIe.

N5, PCIe 4.0, and HBM2e. This chip was probably delayed two years.

Good point, it's built on TSMC while Intel is pushing to become the #2 foundry. Probably it's because Gaudi was made by an Israeli company Intel acquired in 2019 (not an internal project). Who knows.

https://www.semianalysis.com/p/is-intel-back-foundry-and-pro...

Re: Intel Gaudi 3 AI Accelerator

#70

Post by sangnoir »

Scaevolus wrote: post #49
latchkey wrote: post #43

Earlier quoted context omitted.

[flagged]

Effectively everyone that has attempted to use AMD hardware for ML comes away with these opinions, the main difference is how angrily they express it.

So who's buying all the MI300s? Groq seems to be fine with AMD.
Post reply on HN