Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel Gaudi 3 AI Accelerator

intel.com

Post reply on HN
Print view View original

91–100 of 260 posts

Re: Intel Gaudi 3 AI Accelerator

#91

Post by JonChesterfield »

bayindirh wrote: post #61
neilmovva wrote: post #12

A bit surprised that they're using HBM2e, which is what Nvidia A100 (80GB) used back in 2020. But Intel is using 8 stacks here, so Gaudi 3 achieves comparable total bandwidth (3.7TB/s) to H100 (3.4TB/s) which uses 5 stacks of HBM3. Hopefully the older HBM has better supply - HBM3 is hard to get right now! The Gaudi 3 multi-chip package also looks interesting. I see 2 central compute dies, 8 HBM die stacks, and then 6…

> A bit surprised that they're using HBM2e, which is what Nvidia A100 (80GB) used back in 2020. This is one of the secret recipes of Intel. They can use older tech and push it a little further to catch/surpass current gen tech until current gen becomes easier/cheaper to produce/acquire/integrate. They have done it with their first quad core processors by merging two dual core processors (Q6xxx series), or by creating…

> This is one of the secret recipes of Intel

Any other examples of this? I remember the secret sauce being a process advantage over the competition, exactly the opposite of making old tech outperform the state of the art.

Re: Intel Gaudi 3 AI Accelerator

#92

Post by JonChesterfield »

alexey-salmin wrote: post #78
bayindirh wrote: post #61

Earlier quoted context omitted.

> A bit surprised that they're using HBM2e, which is what Nvidia A100 (80GB) used back in 2020. This is one of the secret recipes of Intel. They can use older tech and push it a little further to catch/surpass current gen tech until current gen becomes easier/cheaper to produce/acquire/integrate. They have done it with their first quad core processors by merging two dual core processors (Q6xxx series), or by creating…

Oh dear, Q6600 was so bad, I regret ever owning it

65nm tolerated a lot of voltage. Fun thing to overclock.

Re: Intel Gaudi 3 AI Accelerator

#93

Post by latchkey »

tucnak wrote: post #81
sangnoir wrote: post #70

Earlier quoted context omitted.

So who's buying all the MI300s? Groq seems to be fine with AMD.

Groq doesn't use AMD afaik, they had designed hardware of their own, which is actually 1000s of SRAM chips in a trench-coat.

Speaking of SRAM, I found this relevant comment insightful:

https://news.ycombinator.com/item?id=39966620

Re: Intel Gaudi 3 AI Accelerator

#94

Post by bayindirh »

alexey-salmin wrote: post #78
bayindirh wrote: post #61

Earlier quoted context omitted.

> A bit surprised that they're using HBM2e, which is what Nvidia A100 (80GB) used back in 2020. This is one of the secret recipes of Intel. They can use older tech and push it a little further to catch/surpass current gen tech until current gen becomes easier/cheaper to produce/acquire/integrate. They have done it with their first quad core processors by merging two dual core processors (Q6xxx series), or by creating…

Oh dear, Q6600 was so bad, I regret ever owning it

I owned one, it was a performant little chip. Developed my first multi core stuff with it.

I loved it, to be honest.

Re: Intel Gaudi 3 AI Accelerator

#95

Post by JonChesterfield »

tucnak wrote: post #37
Hugsun wrote: post #25

Earlier quoted context omitted.

Does it not work for them? Where can I learn why?

Just go have a look around Github issues in their ROCm repositories on Github. A few months back the top excuse re: AMD was that we're not supposed to use their "consumer" cards, however the datacenter stuff is kosher. Well, guess what, we have purchased their datacenter card, MI50, and it's similarly screwed. Too many bugs in the kernel, kernel crashes, hangs, and the ROCm code is buggy / incomplete. When it works,…

We absolutely hammered the MI50 in internal testing for ages. Was solid as far as I can tell.

Rocm is sensitive to matching kernel version to driver version to userspace version. Staying very much on the kernel version from a official release and using the corresponding driver is drastically more robust than optimistically mixing different components. In particular, rocm is released and tested as one large blob, and running that large blob on a slightly different kernel version can go very badly. Mixing things from GitHub with things from your package manager is also optimistic.

Imagine it as huge ball of code where cross version compatibility of pieces is totally untested.

Re: Intel Gaudi 3 AI Accelerator

#96

Post by calaphos »

JonChesterfield wrote: post #91
bayindirh wrote: post #61

Earlier quoted context omitted.

> A bit surprised that they're using HBM2e, which is what Nvidia A100 (80GB) used back in 2020. This is one of the secret recipes of Intel. They can use older tech and push it a little further to catch/surpass current gen tech until current gen becomes easier/cheaper to produce/acquire/integrate. They have done it with their first quad core processors by merging two dual core processors (Q6xxx series), or by creating…

> This is one of the secret recipes of Intel Any other examples of this? I remember the secret sauce being a process advantage over the competition, exactly the opposite of making old tech outperform the state of the art.

Intels surprisingly fast 14nm processors come to mind. Born of necessity as they couldn't get their 10 and later 7nm processes working for years. Despite that Intel managed to keep up in single core performance with newer 7nm AMD chips, although at a mich higher power draw.

Re: Intel Gaudi 3 AI Accelerator

#97

Post by timr »

JonChesterfield wrote: post #91
bayindirh wrote: post #61

Earlier quoted context omitted.

> A bit surprised that they're using HBM2e, which is what Nvidia A100 (80GB) used back in 2020. This is one of the secret recipes of Intel. They can use older tech and push it a little further to catch/surpass current gen tech until current gen becomes easier/cheaper to produce/acquire/integrate. They have done it with their first quad core processors by merging two dual core processors (Q6xxx series), or by creating…

> This is one of the secret recipes of Intel Any other examples of this? I remember the secret sauce being a process advantage over the competition, exactly the opposite of making old tech outperform the state of the art.

Back in the day, Intel was great for overclocking because all of their chips could run at significantly higher speeds and voltages than on the tin. This was because they basically just targeted the higher specs, and sold the underperforming silicon as lower-tier products.

Don't know if this counts, but feels directionally similar.

Re: Intel Gaudi 3 AI Accelerator

#98

Post by chucke1992 »

alexey-salmin wrote: post #78
bayindirh wrote: post #61

Earlier quoted context omitted.

> A bit surprised that they're using HBM2e, which is what Nvidia A100 (80GB) used back in 2020. This is one of the secret recipes of Intel. They can use older tech and push it a little further to catch/surpass current gen tech until current gen becomes easier/cheaper to produce/acquire/integrate. They have done it with their first quad core processors by merging two dual core processors (Q6xxx series), or by creating…

Oh dear, Q6600 was so bad, I regret ever owning it

Q6600 was quite good but E8400 was the best.

Re: Intel Gaudi 3 AI Accelerator

#99

Post by chucke1992 »

bayindirh wrote: post #80
mvkel wrote: post #67

Earlier quoted context omitted.

Interesting. Would you say this means Intel is "back," or just not completely dead?

No, this means Intel has woken up and trying. There's no guarantee in anything. I'm more of an AMD person, but I want to see fierce competition, not monopoly, even if it's "my team's monopoly".

Well the only reason why AMD is doing good at CPU is becoming Intel is sleeping. Otherwise it would be Nvidia vs AMD (less steroids though).

Re: Intel Gaudi 3 AI Accelerator

#100

Post by sliken »

SteveNuts wrote: post #64
gnabgib wrote: post #5

Earlier quoted context omitted.

Bit of an embarrassing typo, they do later qualify it as 3.7TB/s

Most of the time bandwidth is expressed in giga/gibi/tera/tebi bits per second so this is also confusing to me

Only for networking, not for anything measured inside a node. Disk bandwidth, cache bandwidth, and memory bandwidth is nearly always measured in bytes/sec (bandwidth), or NS/cache line or similar (which is mix of bandwidth and latency).
Post reply on HN