Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel Gaudi 3 AI Accelerator

intel.com

Post reply on HN
Print view View original

121–130 of 260 posts

Re: Intel Gaudi 3 AI Accelerator

#121

Post by wmf »

mk_stjames wrote: post #111

One nice thing about this (and the new offerings from AMD) is that they will be using the "open accelerator module (OAM)" interface- which standardizes the connector that they use to put them on baseboards, similar to the SXM connections of Nvidia that use MegArray connectors to thier baseboards. With Nvidia, the SXM connection pinouts have always been held proprietary and confidential. For example, P100's and V100's…

Why don't they sell used P100 DGX/HGX servers as a unit? Are those bare P100s only so cheap precisely because they're useless?

Re: Intel Gaudi 3 AI Accelerator

#122

Post by wmf »

ZoomerCretin wrote: post #106
geertj wrote: post #103

I wonder if someone knowledgeable could comment on OneAPI vs Cuda. I feel like if Intel is going to be a serious competitor to Nvidia, both software and hardware are going to be equally important.

I'm not familiar with the particulars of OneAPI, but it's just a matter of rewriting CUDA kernels into OneAPI. This is pretty trivial for the vast majority of small (<5 LoC) kernels. Unlike AMD, it looks like they're serious about dogfooding their own chips, and they have a much better reputation for their driver quality.

IMO dogfooding Gaudi would mean training a model on it (and the only way to "prove" it would be to release that model).

Re: Intel Gaudi 3 AI Accelerator

#123

Post by AnthonyMouse »

tormeh wrote: post #101
0cf8612b2e1e wrote: post #58

Earlier quoted context omitted.

Surely NVidia’s pricing is more what the market will bear vs an intrinsic cost to build. Intel being the underdog should be willing to offer a discount just to get their foot in the door.

Pricing is normally what the market will bear. If this is below your cost as supplier you exit the market.

But if your competitor's price is dramatically above your cost, you can provide a huge discount as an incentive for customers to pay the transition cost to your system while still turning a tidy profit.

Re: Intel Gaudi 3 AI Accelerator

#124

Post by wmf »

Manabu-eo wrote: post #88
blackeyeblitzar wrote: post #17

Earlier quoted context omitted.

See https://www.nextplatform.com/2024/04/09/with-gaudi-3-intel-c... for more details. Here’s the relevant bits, although you should visit the article to see the networking diagrams: > The Gaudi 3 accelerators inside of the nodes are connected using the same OSFP links to the outside world as happened with the Gaudi 2 designs, but in this case the doubling of the speed means that Intel has had to add retimers between…

The PCI-e HL-338 version is also listing 24 200GbE RDMA nics in a dual-slot configuration. How would they be connected?

They may go to the top of the card where you can use an SLI-like bridge to connect multiple cards.

Re: Intel Gaudi 3 AI Accelerator

#125

Post by JonChesterfield »

colechristensen wrote: post #9

Anyone have experience and suggestions for an AI accelerator? Think prototype consumer product with total cost preferably < $500, definitely less than $1000.

I liked my 5700XT. That seems to be $200 now. Ran arbitrary code on it just fine. Lots of machine learning seems to be obsessed with amount of memory though and increasing that is likely to increase the price. Also HN doesn't like ROCm much, so there's that.

Re: Intel Gaudi 3 AI Accelerator

#126

Post by nullindividual »

brookst wrote: post #13
riskable wrote: post #7

> Twenty-four 200 gigabit (Gb) Ethernet ports are integrated into every Intel Gaudi 3 accelerator WHAT‽ It's basically got the equivalent of a 24-port, 200-gigabit switch built into it. How does that make sense? Can you imaging stringing 24 Cat 8 cables between servers in a single rack? Wait: How do you even decide where those cables go? Do you buy 24 Gaudi 3 accelerators and run cables directly between every single…

Audio folks solved the "which cable goes where" problem ages ago with cable snakes: https://www.seismicaudiospeakers.com/products/24-channel-xlr... But I'm not how big and how expensive a 24 channel cat 8 snake would be (!).

I wouldn’t think that would be appropriate for Ethernet due to cross talk.

Re: Intel Gaudi 3 AI Accelerator

#127

Post by VHRanger »

astrodust wrote: post #116
kylixz wrote: post #53

This is a bit snarky — but will Intel actually keep this product line alive for more than a few years? Having been bitten by building products around some of their non-x86 offerings where they killed good IP off and then failed to support it… I’m skeptical. I truly do hope it is successful so we can have some alternative accelerators.

I hope it pairs well with Optane modules!

I'll add it right next to my Xeon Phi!

Re: Intel Gaudi 3 AI Accelerator

#129

Post by tucnak »

JonChesterfield wrote: post #95
tucnak wrote: post #37

Earlier quoted context omitted.

Just go have a look around Github issues in their ROCm repositories on Github. A few months back the top excuse re: AMD was that we're not supposed to use their "consumer" cards, however the datacenter stuff is kosher. Well, guess what, we have purchased their datacenter card, MI50, and it's similarly screwed. Too many bugs in the kernel, kernel crashes, hangs, and the ROCm code is buggy / incomplete. When it works,…

We absolutely hammered the MI50 in internal testing for ages. Was solid as far as I can tell. Rocm is sensitive to matching kernel version to driver version to userspace version. Staying very much on the kernel version from a official release and using the corresponding driver is drastically more robust than optimistically mixing different components. In particular, rocm is released and tested as one large blob, and…

I would run simple llama.cpp batch jobs for 10 minutes when it would suddenly fail, and require a restart. Random VM_L2_PROTECTION_FAULT in dmesg, something having to do with doorbells. I did report this, never heard back from them.

Re: Intel Gaudi 3 AI Accelerator

#130

Post by formerly_proven »

mk_stjames wrote: post #111

One nice thing about this (and the new offerings from AMD) is that they will be using the "open accelerator module (OAM)" interface- which standardizes the connector that they use to put them on baseboards, similar to the SXM connections of Nvidia that use MegArray connectors to thier baseboards. With Nvidia, the SXM connection pinouts have always been held proprietary and confidential. For example, P100's and V100's…

The price is low because they’re useless (except for replacing dead cards in a DGX), if you had a 40$ PCIe AIC-to-SXM adapter, the price would go up a lot.

> I'm one of those people who finds 'retro-super-computing' a cool hobby and thus the interfaces like OAM being open means that these devices may actually have a life for hobbyists in 8~10 years instead of being sent directly to the bins due to secret interfaces and obfuscated backplane specifications.

Very cool hobby. It’s also unfortunate how stringent e-waste rules lead to so much perfectly fine hardware to be scrapped. And how the remainder is typically pulled apart to the board / module level for spares. Makes it very unlikely to stumble over more or less complete-ish systems.

Post reply on HN