Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel Gaudi 3 AI Accelerator

intel.com

Post reply on HN
Print view View original

141–150 of 260 posts

Re: Intel Gaudi 3 AI Accelerator

#141

Post by p1esk »

chessgecko wrote: post #51
wongarsu wrote: post #35

Earlier quoted context omitted.

Probably not. An 40GB Nvidia A100 is arguably reasonable for a workstation at $6000. Depending on your definition an 80GB A100 for $16000 is still reasonable. I don't see this being cheaper than an 80GB A100. Probably a good bit more expensive, seeing as it has more RAM, compares itself favorably to the H100, and has enough compelling features that it probably doesn't have to (strongly) compete on price.

I think you're right on the price, but just to give some false hope. I think newish hbm (and this is hbm2e which is a little older) is around $15/gb so for 128 gb thats $1920. There are some other cogs, but in theory they could sell this for like $3-4k and make some gross profit while getting some hobbyist mindshare/research code written for it. I doubt they will though, it might eat too much into profits from the no…

in theory they could sell this for like $3-4k

You’re joking, right? They will price it to match current H100 pricing. Multiply your estimate by 10x.

Re: Intel Gaudi 3 AI Accelerator

#142

Post by mk_stjames »

wmf wrote: post #121
mk_stjames wrote: post #111

One nice thing about this (and the new offerings from AMD) is that they will be using the "open accelerator module (OAM)" interface- which standardizes the connector that they use to put them on baseboards, similar to the SXM connections of Nvidia that use MegArray connectors to thier baseboards. With Nvidia, the SXM connection pinouts have always been held proprietary and confidential. For example, P100's and V100's…

Why don't they sell used P100 DGX/HGX servers as a unit? Are those bare P100s only so cheap precisely because they're useless?

I have a theory some big cloud provider moved a ton of racks from SXM2 P100's to SXM2 V100's (those were a thing) and thus orphaned an absolute ton of P100's without their baseboards.

Or, these salvage operations just stripped racks and kept the small stuff and e-waste the racks because they think it's the more efficient use of their storage space and would be easier to sell, without thinking correctly.

Re: Intel Gaudi 3 AI Accelerator

#143

Post by narrator »

wongarsu wrote: post #35
sairahul82 wrote: post #18

Can we expect the price of 'Gaudi 3 PCIe' to be reasonable enough to put in a workstation? That would be a game changer for local LLMs

Probably not. An 40GB Nvidia A100 is arguably reasonable for a workstation at $6000. Depending on your definition an 80GB A100 for $16000 is still reasonable. I don't see this being cheaper than an 80GB A100. Probably a good bit more expensive, seeing as it has more RAM, compares itself favorably to the H100, and has enough compelling features that it probably doesn't have to (strongly) compete on price.

Isn't it much better to get a Mac Studio with an M2 Max and 192gb of Ram and 31 terraflops for $6599 and run llama.cpp?

Re: Intel Gaudi 3 AI Accelerator

#145

Post by deepnotderp »

calaphos wrote: post #96
JonChesterfield wrote: post #91

Earlier quoted context omitted.

> This is one of the secret recipes of Intel Any other examples of this? I remember the secret sauce being a process advantage over the competition, exactly the opposite of making old tech outperform the state of the art.

Intels surprisingly fast 14nm processors come to mind. Born of necessity as they couldn't get their 10 and later 7nm processes working for years. Despite that Intel managed to keep up in single core performance with newer 7nm AMD chips, although at a mich higher power draw.

That's because CPU performance cares less about transistor density and more about transistor performance, and 14nm drive strength was excellent

Re: Intel Gaudi 3 AI Accelerator

#146

Post by iamleppert »

kylixz wrote: post #53

This is a bit snarky — but will Intel actually keep this product line alive for more than a few years? Having been bitten by building products around some of their non-x86 offerings where they killed good IP off and then failed to support it… I’m skeptical. I truly do hope it is successful so we can have some alternative accelerators.

Long enough for you to get in, develop some AI product, raise investment funds, and get out with your bag!

Re: Intel Gaudi 3 AI Accelerator

#147

Post by jmward01 »

latchkey wrote: post #83
jmward01 wrote: post #75

Earlier quoted context omitted.

Yeah, this has stopped me from trying anything with them. They need to lead with their consumer cards so that developers can test/build/evaluate/gain trust locally and then their enterprise offerings need to 100% guarantee that the stuff developers worked on will work in the data center. I keep hoping to see this but every time I look it isn't there. There is way more support for apple silicon out there than ROCm and…

You are right, AMD should do more with consumer cards, but I understand why they aren't today. It is a big ship, they've really only started changing course as of last Oct/Nov, before the release of MI300x in Dec. If you have limited resources and a whole culture to change, you have to give them time to fix that. That said, if you're on the inside, like I am, and you talk to people at AMD (just got off two separate b…

I look forward to seeing it. NVIDIA needs real competition for their own benefit if not the market as a whole. I want a richer ecosystem where Intel, AMD, NVIDIA and other players all join in with the winner being the consumer. From a selfish point of view I also want to do more home experimentation. LLMs are so new that you can make breakthroughs without a huge team but it really helps to have hardware to make it easier to play with ideas. Consumer card memory limitations are hurting that right now.

Re: Intel Gaudi 3 AI Accelerator

#148

Post by ZoomerCretin »

JonChesterfield wrote: post #118
ZoomerCretin wrote: post #106

Earlier quoted context omitted.

I'm not familiar with the particulars of OneAPI, but it's just a matter of rewriting CUDA kernels into OneAPI. This is pretty trivial for the vast majority of small (<5 LoC) kernels. Unlike AMD, it looks like they're serious about dogfooding their own chips, and they have a much better reputation for their driver quality.

All the dev work at AMD is on our own hardware. Even things like the corporate laptops are ryzen based. The first gen ryzen laptop I got was terrible but it wasn't intel. We also do things like develop ROCm on the non-qualified cards and build our tools with our tools. It would be crazy not to.

Yes that's why I qualified "serious" dogfooding. Of course you use your hardware for your own development work, but it's clearly not enough given that showstopper driver issues are going unfixed for half a year.

Re: Intel Gaudi 3 AI Accelerator

#149

Post by TApplencourt »

alecco wrote: post #120
ZoomerCretin wrote: post #106

Earlier quoted context omitted.

I'm not familiar with the particulars of OneAPI, but it's just a matter of rewriting CUDA kernels into OneAPI. This is pretty trivial for the vast majority of small (<5 LoC) kernels. Unlike AMD, it looks like they're serious about dogfooding their own chips, and they have a much better reputation for their driver quality.

Trivial??

You have SYCLomatic to help.

Re: Intel Gaudi 3 AI Accelerator

#150

Post by ZoomerCretin »

alecco wrote: post #120
ZoomerCretin wrote: post #106

Earlier quoted context omitted.

I'm not familiar with the particulars of OneAPI, but it's just a matter of rewriting CUDA kernels into OneAPI. This is pretty trivial for the vast majority of small (<5 LoC) kernels. Unlike AMD, it looks like they're serious about dogfooding their own chips, and they have a much better reputation for their driver quality.

Trivial??

That statement has two qualifications.
Post reply on HN