Intel Gaudi 3 AI Accelerator • Hacker News Board

Re: Intel Gaudi 3 AI Accelerator

#151

Post by latchkey » Tue, Apr 09, 2024, 9:56 PM UTC

jmward01 wrote: post #147

latchkey wrote: post #83

Earlier quoted context omitted.

You are right, AMD should do more with consumer cards, but I understand why they aren't today. It is a big ship, they've really only started changing course as of last Oct/Nov, before the release of MI300x in Dec. If you have limited resources and a whole culture to change, you have to give them time to fix that. That said, if you're on the inside, like I am, and you talk to people at AMD (just got off two separate b…

I look forward to seeing it. NVIDIA needs real competition for their own benefit if not the market as a whole. I want a richer ecosystem where Intel, AMD, NVIDIA and other players all join in with the winner being the consumer. From a selfish point of view I also want to do more home experimentation. LLMs are so new that you can make breakthroughs without a huge team but it really helps to have hardware to make it ea…

> I want a richer ecosystem where Intel, AMD, NVIDIA and other players all join in with the winner being the consumer.

This is exactly the void I'm trying to fill.

Re: Intel Gaudi 3 AI Accelerator

#152

Post by mk_stjames » Tue, Apr 09, 2024, 10:07 PM UTC

buildbot wrote: post #138

mk_stjames wrote: post #111

One nice thing about this (and the new offerings from AMD) is that they will be using the "open accelerator module (OAM)" interface- which standardizes the connector that they use to put them on baseboards, similar to the SXM connections of Nvidia that use MegArray connectors to thier baseboards. With Nvidia, the SXM connection pinouts have always been held proprietary and confidential. For example, P100's and V100's…

The SXM2 interface is actually publicly documented! There is an open compute spec for a 8-way baseboard. You can find the pinouts there.

I had read their documents such as the spec for the Big Basin JBOG, where everything is documented except the actual pinouts on the base board. Everything leading up to it and from it is there but the actual MegArray pinout connection to a single P100/V100 I never found.

But maybe there was more I missed. I'll take another look.

Re: Intel Gaudi 3 AI Accelerator

#153

Post by InitEnabler » Tue, Apr 09, 2024, 10:09 PM UTC

amelius wrote: post #136

Missing in these pictures are the thermal management solutions.

If you look at one of the pictures you can get a peak at what they look like (I think...) in the bottom right.

https://www.intel.com/content/dam/www/central-libraries/us/e...

Re: Intel Gaudi 3 AI Accelerator

#154

Post by prewett » Tue, Apr 09, 2024, 10:11 PM UTC

TheAceOfHearts wrote: post #68

whalesalad wrote: post #31

https://www.merriam-webster.com/dictionary/gaudy

Honestly, I thought the same thing upon reading the name. I'm aware of the reference to Antoni Gaudí, but having the name sound so close to gaudy seems a bit unfortunate. Surely they must've had better options? Then again I don't know how these sorts of names get decided anymore.

'Gaudi' is properly pronounced Ga-oo-DEE in his native Catalan, whereas (in my dialect) 'gaudy' is pronounced GAW-dee. My guess is Intel wasn't even thinking about 'gaudy' because they were thinking about "famous architects" or whatever the naming pool was. Although, I had heard that the 'gaudy' came from the architect's name because of what people thought of his work. (I'm not sure this is correct, it was just my first introduction to the word.)

Re: Intel Gaudi 3 AI Accelerator

#155

Post by ipsum2 » Tue, Apr 09, 2024, 10:14 PM UTC

sairahul82 wrote: post #18

Can we expect the price of 'Gaudi 3 PCIe' to be reasonable enough to put in a workstation? That would be a game changer for local LLMs

It won't be under $10k.

Re: Intel Gaudi 3 AI Accelerator

#156

Post by chessgecko » Tue, Apr 09, 2024, 10:15 PM UTC

p1esk wrote: post #141

chessgecko wrote: post #51

Earlier quoted context omitted.

I think you're right on the price, but just to give some false hope. I think newish hbm (and this is hbm2e which is a little older) is around $15/gb so for 128 gb thats $1920. There are some other cogs, but in theory they could sell this for like $3-4k and make some gross profit while getting some hobbyist mindshare/research code written for it. I doubt they will though, it might eat too much into profits from the no…

in theory they could sell this for like $3-4k You’re joking, right? They will price it to match current H100 pricing. Multiply your estimate by 10x.

They could, I know they wont, but they wouldn't lose money on the parts

Re: Intel Gaudi 3 AI Accelerator

#157

Post by sorenjan » Tue, Apr 09, 2024, 10:40 PM UTC

JonChesterfield wrote: post #118

ZoomerCretin wrote: post #106

Earlier quoted context omitted.

I'm not familiar with the particulars of OneAPI, but it's just a matter of rewriting CUDA kernels into OneAPI. This is pretty trivial for the vast majority of small (<5 LoC) kernels. Unlike AMD, it looks like they're serious about dogfooding their own chips, and they have a much better reputation for their driver quality.

All the dev work at AMD is on our own hardware. Even things like the corporate laptops are ryzen based. The first gen ryzen laptop I got was terrible but it wasn't intel. We also do things like develop ROCm on the non-qualified cards and build our tools with our tools. It would be crazy not to.

Why isn't AMD part of the UXL Foundation? What does AMD gain from not working together with other companies do make an open alternative to Cuda?

Please make SYCL a priority, cross platform code would make AMD GPUs a viable alternative in the future.

Re: Intel Gaudi 3 AI Accelerator

#158

Post by wmf » Tue, Apr 09, 2024, 10:43 PM UTC

nullindividual wrote: post #126

brookst wrote: post #13

Earlier quoted context omitted.

Audio folks solved the "which cable goes where" problem ages ago with cable snakes: https://www.seismicaudiospeakers.com/products/24-channel-xlr... But I'm not how big and how expensive a 24 channel cat 8 snake would be (!).

I wouldn’t think that would be appropriate for Ethernet due to cross talk.

Four-lane and eight-lane twinax cables exist; I think each pair is individually shielded. Beyond that there's fiber.

Re: Intel Gaudi 3 AI Accelerator

#159

Post by pezezin » Tue, Apr 09, 2024, 10:47 PM UTC

nullindividual wrote: post #126

brookst wrote: post #13

Earlier quoted context omitted.

Audio folks solved the "which cable goes where" problem ages ago with cable snakes: https://www.seismicaudiospeakers.com/products/24-channel-xlr... But I'm not how big and how expensive a 24 channel cat 8 snake would be (!).

I wouldn’t think that would be appropriate for Ethernet due to cross talk.

Those cables definitely exist for Ethernet, and regarding cross talk, that's what shielding is for.

Although not for 200 Gbps, at that rate you either use big twinax DACs, or go to fibre.

Re: Intel Gaudi 3 AI Accelerator

#160

Post by 01HNNWZ0MV43FF » Tue, Apr 09, 2024, 11:02 PM UTC

JonChesterfield wrote: post #115

mk_stjames wrote: post #111

One nice thing about this (and the new offerings from AMD) is that they will be using the "open accelerator module (OAM)" interface- which standardizes the connector that they use to put them on baseboards, similar to the SXM connections of Nvidia that use MegArray connectors to thier baseboards. With Nvidia, the SXM connection pinouts have always been held proprietary and confidential. For example, P100's and V100's…

I really like this side to AMD. There's a strategic call somewhere high up to bias towards collaboration with other companies. Sharing the fabric specifications with broadcom was an amazing thing to see. It's not out of the question that we'll see single chips with chiplets made by different companies attached together.

Maybe they feel threatened by ARM on mobile and Intel on desktop / server. Companies that think they're first try to monopolize. Companies that think they're second try to cooperate.