Live data from Hacker News

Intel Gaudi 3 AI Accelerator

intel.com

241–250 of 260 posts

Re: Intel Gaudi 3 AI Accelerator

#241
post #173

Earlier quoted context omitted.

Habana was an acquisition and their use of TSMC predates the acquisition.

Yeah, but if Intel can't even get internal customers to adopt their foundry services it seems to bode poorly for the future of the company.

The design and decision to make it Fab with TSMC was way ahead of Intel's Foundry services offering. ( And it is not like Intel had the extra capacity planned at the time for Intel's GPU )

Re: Intel Gaudi 3 AI Accelerator

#242
post #173

Earlier quoted context omitted.

Habana was an acquisition and their use of TSMC predates the acquisition.

Yeah, but if Intel can't even get internal customers to adopt their foundry services it seems to bode poorly for the future of the company.

Process matters. Intel was ahead for a long time, and has been behind for a long time. Perhaps they will be ahead again, but maybe not. I’d rather see them competitive.

Re: Intel Gaudi 3 AI Accelerator

#243

Earlier quoted context omitted.

Isn't it much better to get a Mac Studio with an M2 Max and 192gb of Ram and 31 terraflops for $6599 and run llama.cpp?

Macs don't support CUDA which means all that wonderful hardware will be useless when trying to do anything with AI for at least a few years. There's Metal but it has its own set of problems, biggest one being it isn't a drop in CUDA replacement.

You can do LLM inference without CUDA just fine. Download Ollama and see for yourself

Re: Intel Gaudi 3 AI Accelerator

#244

Earlier quoted context omitted.

Only "most of the time" ? :( You'd hope at $15,000+ per unit, you wouldn't have to reset it at all...

It's $1000 per, no? This is one of the gaming cards.

Yep, bought for $1000.

At which price point to be honest, it still shouldn't be needed.

AMD are lucky everyone expects this nowadays, or people might consider sueing.

Re: Intel Gaudi 3 AI Accelerator

#245

Earlier quoted context omitted.

I'm not familiar with the particulars of OneAPI, but it's just a matter of rewriting CUDA kernels into OneAPI. This is pretty trivial for the vast majority of small (<5 LoC) kernels. Unlike AMD, it looks like they're serious about dogfooding their own chips, and they have a much better reputation for their driver quality.

(reply to Zoomer from further down, moving up because I ended up writing a lot) This experience is largely a misalignment between what AMD thinks their product is and what the Linux world thinks software is. My pet theory is it's a holdover from the GPU being primarily a games console product as that's what kept the company alive through the recent dark times. There's money now but some of the best practices are stic…

Wow thank you for the insight. I appreciate you taking the time to write all of this out, and also for your stubbornness in testing!

Re: Intel Gaudi 3 AI Accelerator

#246

Earlier quoted context omitted.

Upon further review... I think any actual base board schematics / pinouts touching the Nvidia hardware directly is indeed kept behind some sort of NDA or OEM license agreement and is specifically kept out of any of those documents for the Open Compute project JBOG rigs. I think this is literally the impetus for their OAM spec which makes the pinout open and shareable. Up until that, they had to keep the actual design…

Hmm interesting, I was linked to an OCP dropbox with a version that did have the connector pinouts. Maybe something someone shouldn’t have posted then…

couldn't someone just buy one of those chinese sxm2 to pcie adapter boards and test continuity to get the pinouts? I have one that could take like 10 minutes

Re: Intel Gaudi 3 AI Accelerator

#247
post #207

Earlier quoted context omitted.

Itanium only stuck around as long as it did because they were obligated to support HP.

Itanium only failed because AMD was allowed to come up with AMD64, Intel would have managed to push Itanium no matter what, if there were no alternatives to a 64bit compatible x86 CPU.

Itanium wasn't x86 compatible, it used the EPIC VLIW instruction set. It relied heavily on compiler optimization that never really materialized. I think it was called speculative precompilation or something like that. The Itanium suffered in two ways that had interplay with one another. The first is that it was very latency sensitive and non-deterministic fetches stalled it. The second was there often weren't enough parallel instructions to execute simultaneously. In both cases the processor spent a lot of time executing NOPs.

Modern CPUs have moved towards becoming simpler and more flexible in their execution with specialized hardware (GPUs, etc) for the more parallel and repetitive tasks that Itanium excelled at.

Re: Intel Gaudi 3 AI Accelerator

#248
post #228

Earlier quoted context omitted.

Did you run on the blessed Ubuntu version with the blessed kernel version and the blessed driver version? As otherwise you really are in a development branch. If you can point me to a repro I'll add it to my todo list. You can probably tag me in the github issue if that's where you reported it.

> blessed Ubuntu version with the blessed kernel version To an SRE, this is a nightmare to read. Cuda is bad in this regard (can often prevent major kernel version updates), but this is worse.

I feel like this goes both ways. You also don't want to have to run bleeding edge for everything because there are so many bugs in things. You kind of want known stable versions to at least base yourself off of.

Re: Intel Gaudi 3 AI Accelerator

#249

> Twenty-four 200 gigabit (Gb) Ethernet ports are integrated into every Intel Gaudi 3 accelerator How much does a single 200Gbit active (or inactive) fiber cable cost? Probably thousands of dollars.. making even the cabling for each card Very Expensive. Nevermind the network switches themselves.. Simultaneously impressive and disappointing.

What do you mean by active vs inactive fiber cable? I tried to Google about this distinction, but I couldn't find anything helpful.

My off-the-cuff take: AOC's are a specific kind of fiber optic cable, typically used in data center applications for 100Gbit+ connections. The alternate types of fiber are typically referred to as passive fiber cables, e.g. simplex or duplex, single-mode (single fiber strands, usually in a yellow jacket) or multi-mode (multiple fiber strands, usually in an orange jacket). Each type of passive fiber cable has specific applications and requires matching transceivers, whereas AOCs are self-contained with the transceivers pre-terminated on.

If you search for "AOC Fiber", lots of resources will pop up. FS.com is one helpful resource.

https://community.fs.com/article/active-optical-cable-aoc-ri...

> Active optical cable (AOC) can be defined as an optical fiber jumper cable terminated with optical transceivers on both ends. It uses electrical-to-optical conversion on the cable ends to improve speed and distance performance of the cable without sacrificing compatibility with standard electrical interfaces.

Re: Intel Gaudi 3 AI Accelerator

#250
post #166

Earlier quoted context omitted.

Well, lets not forget, AMD is AMD because they reverse-engineered Intel chips....

IBM didn't want to rely solely on Intel when introducing PCs so it forced Intel to share its arch with another manufacturer that turned out to be AMD. It's not like AMD stole it. Math coprocessor was in turn invented by AMD (Am9511, Am9512) and licensed by Intel (8231, 8232).

Also AMD64
Post reply on HN