Live data from Hacker News

Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild

servethehome.com

171–180 of 188 posts

Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild

#171
post #157
post #36

Earlier quoted context omitted.

I don't think we'll be legally prohibited from buying it but there will be zero English docs (see Allwinner and such). Maybe if you're lucky you'll get an uncommented code dump with a forked years-old version of PyTorch.

Can't an LLM translate?

Touche.

Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild

#172
post #6

About time. NVIDIA needs some serious competition.

Did you watch the Nvidia 2024 GTC keynote[1]?

The CEO of NVIDIA was pretty clear: Nvidia's focus is now on selling Blackwell AI Factories. According to some napkin math, each of these Blackwell AI factories will have an annual power bill of roughly $8,000,000 USD and if you do a little apples to oranges comparison will outperform the computer that's currently #1 on the supercomputing top 500 (Frontier at Oak Ridge) by one to two orders of magnitude.

for comparison, over the past fifteen years, we've seen 3 orders of magnitude improvement in the supercomputing top 500. (from roadrunner in 2008 to frontier in 2024). Nvidia is going to do 1 to 2 orders of magnitude improvement instantaneously.

Nvidia's core market has shifted dramatically. There is no competition.

[1]: https://www.nvidia.com/gtc/keynote/

Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild

#173

Earlier quoted context omitted.

You don’t need to manage NVLink. NVLink either talks native NVlink to itself when you are using NVlink switches either intra-server or intra-rack or; It can talk PCIe over NVlink when talking to a PCIe endpoint. Or you can run Infiniband or Ethernet on top of it and talk to w/e is on the other side. Gaudi isn’t that different remember Ethernet != TCP/IP.

So it works with Ethernet switches?

[deleted]

Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild

#174

Earlier quoted context omitted.

AMD is doing several billion in ai processor sales already and the new chip is selling as fast as they can make them. At least with AMD, a customer can actually get them now as opposed to the nearly 1 year lead time from nVidia.

Now, if they could also do the performant, unified, software and driver part..

This is a big part of why they are generally available. Building on top of AMD gear is currently more difficult than Nvidia. It's unfortunate, but the evolution of/from Cuda when AMD had little in this space gave intel a massive jump-start. As a dev it's way easier to pick up various stacks that work with nvidia gear and not have to worry anywhere near as much about what's underneath.

Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild

#175
post #6

About time. NVIDIA needs some serious competition.

Did you watch the Nvidia 2024 GTC keynote[1]? The CEO of NVIDIA was pretty clear: Nvidia's focus is now on selling Blackwell AI Factories. According to some napkin math, each of these Blackwell AI factories will have an annual power bill of roughly $8,000,000 USD and if you do a little apples to oranges comparison will outperform the computer that's currently #1 on the supercomputing top 500 (Frontier at Oak Ridge) b…

When moving into a market where there are no competitors one should be wary that this may be because there are no customers.

Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild

#176
post #8
post #6

About time. NVIDIA needs some serious competition.

pokes OpenCL's corpse with a stick C'mon, do something...

Its very unfortunate. One of our vendors has spent a hell of an effort making spatial analysis tools work via OpenCL instead of being just processor bound, and its made those industry standard libraries 1-2 orders of magnitude faster. That is something the spatial industry desperately needs in order to improve iteratively.

With the state of OpenCL it's frustrating. So much to be had, but so little improvement and support.

Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild

#178
post #6

About time. NVIDIA needs some serious competition.

The only meaningful hardware competition, meaning lower prices, will come from Chinese designed, Chinese manufactured parts. This is still a long ways out. Is it inevitable? I think so. Before 2019 there wasn't an opportunity, now there is. For software, Chinese universities, Alibaba, Tencent and Bytedace are already releasing models, training code and in rare cases datasets that are competitive with private offering…

Or from Tesla project Dojo

Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild

#179
post #8

Earlier quoted context omitted.

pokes OpenCL's corpse with a stick C'mon, do something...

https://github.com/ROCm/ROCm/issues/2754 Wow and I thought that the latest generation of GPUs was better.

From that exchange, clearly not. They do not have their stuff together. This is one area where AMD really should be employing quite a full team to ensure that major projects, including drivers, are able to support AMD cards without any friction. Whilst it's still a PITA to work.with AMD hardware in most AI frameworks Nvidia will continue to dominate.

Re: Intel Gaudi 3 the New 128GB HBM2e AI Chip in the Wild

#180
post #175

Earlier quoted context omitted.

Did you watch the Nvidia 2024 GTC keynote[1]? The CEO of NVIDIA was pretty clear: Nvidia's focus is now on selling Blackwell AI Factories. According to some napkin math, each of these Blackwell AI factories will have an annual power bill of roughly $8,000,000 USD and if you do a little apples to oranges comparison will outperform the computer that's currently #1 on the supercomputing top 500 (Frontier at Oak Ridge) b…

When moving into a market where there are no competitors one should be wary that this may be because there are no customers.

That's not very convincing against Nvidia. Since they single handedly created the modern AI market through CUDA.
Post reply on HN