Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers

stability.ai

Post reply on HN
Print view View original

41–50 of 66 posts

Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers

#41

Post by david-gpu »

yukIttEft wrote: post #38

I'm wondering how AI scientists work these days. Do they really hack Cudakernels or do they plug models together with highlevel toolkits like pytorch? Considering its the latter, considering pytorch takes care of providing optimized backends for various hardwares, how big of a moat is Cuda then really?

Pytorch relies heavily on the extensive libraries of high-performance kernels provided by NVidia, such as cuDNN.

In other words, it goes something like this:

    Application
    Pytorch (and similar)
    cuDNN (and similar)
    CUDA (and similar)
    NVidia GPU
My opinion, based on what I saw those wizards do, is that reproducing the feature set and efficiency of cuDNN/cuBLAS is deeply nontrivial.

Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers

#42

Post by cherryteastain »

georgeburdell wrote: post #40
cherryteastain wrote: post #39

One question I have that nobody, including an Intel AXG employee, has been able to answer satisfsctorily for me is why both Gaudi and Ponte Vecchio exist. Wouldn't Intel have better chances of success if they focused on one product line?

Gaudi was brought into Intel via an acquisition. Ponte Vecchio was an internal program. It can be explained by a combination of management silos and perhaps pre-existing obligations for Ponte Vecchio with the government for how they both came into being

Sad if it's a management/politics happenstance than a genius master plan.

Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers

#43

Post by mistrial9 »

https://es.wikipedia.org/wiki/Antoni_Gaud%C3%AD

Gaudi is a famous name for a reason.. the flowing lines and frankly, nonsense and silliness, in the art and architecture of Gaudi stands for generations as a contrast to the relentless severity of formal classical arts (and especially a contrast to Intel electronic parts).

Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers

#44

Post by isoprophlex »

doctorpangloss wrote: post #37
MasterScrat wrote: post #4

Interesting! this was already the case with TPUs easily beating A100s. We sell Stable Diffusion finetuning on TPUs (dreamlook.ai), people are amazed how fast and cheap we can offer it - but there's no big secret, we just use hardware that's strictly faster and cheaper per unit of work. I expect a new wave of "your task, but on superior hardware" services to crop up with these chips!

But you and I can't buy a TPU. You and I can buy an H100.

Speak for yourself! I can't even afford 1/10th of an H100.

Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers

#45

Post by sabujp »

GaggiX wrote: post #17
Flux159 wrote: post #13

This is nice to foster some competition in hardware for model training, but the availability of these machines seems very limited - I don't think there's any major cloud provider allowing per hour rental of Gaudi2 VMs and Intel's own site directs you to buy an 8x GPU provisioned server from Supermicro for more than 40k USD. Availability and software stack is still heavily in Nvidia's favor right now, but maybe by the…

>Intel's own site directs you to buy an 8x GPU provisioned server from Supermicro for more than 40k USD Isn't that the price of a single H100?

what you're telling me is that SMCI doesn't care if NVDA or INTC or even AMD outperforms on their hardware, they'll make axes and shovels for anyone and profit either way :)

Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers

#48

Post by wmf »

1024core wrote: post #25

NVIDIA's profit margin is almost 92% on an H100. I'm surprised more chip companies haven't jumped on a "ML accelerator" bandwagon by now.

There's a dozen AI chips already; how many do you want?

Now working ones is a different story.

Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers

#50

Post by wmf »

cherryteastain wrote: post #39

One question I have that nobody, including an Intel AXG employee, has been able to answer satisfsctorily for me is why both Gaudi and Ponte Vecchio exist. Wouldn't Intel have better chances of success if they focused on one product line?

It's good risk reduction, especially since Ponte Vecchio failed.
Post reply on HN