Some analysis of how and/or why it is able to be 3x faster despite no hardware metric being 3x better would make this actually useful and insightful instead of advertising.
Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers
11–20 of 66 posts
Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers
#12"For Stable Diffusion 3, we measured the training throughput for the 2B Multimodal Diffusion Transformer (MMDiT) architecture model. Gaudi 2 trained images 1.5x faster than the H100-80GB, and 3x faster than A100-80GB GPU’s when scaled up to 32 nodes. "
I can feel the NVDA stock slipping as we speak… It has been amazing watching the groupthink at work on that stock when we just saw the same group do it on TSLA to disastrous effect. A similar no moat situation where they simply can’t imagine competitors ever existing.
My father in law was telling me the other day that he was going buy some NVIDIA stock because it is going to go up to 1,400
Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers
#13This is nice to foster some competition in hardware for model training, but the availability of these machines seems very limited - I don't think there's any major cloud provider allowing per hour rental of Gaudi2 VMs and Intel's own site directs you to buy an 8x GPU provisioned server from Supermicro for more than 40k USD. Availability and software stack is still heavily in Nvidia's favor right now, but maybe by the end of the year that will start changing.
Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers
#14H100 was released almost exactly 1 year ago so I guess it's ok if Intel is now ready to compete with last year's model.
To those commenting about "no moat" remember CUDA is a huge part of it, it's actually HW+SW and both took a decade to mature, together
Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers
#15This is nice to foster some competition in hardware for model training, but the availability of these machines seems very limited - I don't think there's any major cloud provider allowing per hour rental of Gaudi2 VMs and Intel's own site directs you to buy an 8x GPU provisioned server from Supermicro for more than 40k USD. Availability and software stack is still heavily in Nvidia's favor right now, but maybe by the…
Think it'll probably crack on with Gaudi3 at 4x performance, twice VRAM etc later this year.
We found cuda sycl conversion surprisingly good https://www.intel.com/content/www/us/en/developer/articles/t...
Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers
#16Interesting! this was already the case with TPUs easily beating A100s. We sell Stable Diffusion finetuning on TPUs (dreamlook.ai), people are amazed how fast and cheap we can offer it - but there's no big secret, we just use hardware that's strictly faster and cheaper per unit of work. I expect a new wave of "your task, but on superior hardware" services to crop up with these chips!
The v5es and v5ps are pretty amazing at running SD, giving code for SD3 now to optimise it on those.
v5es are particularly interesting given the millions that will land and the large pod sizes, particularly well constructed for million token context windows.
Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers
#17This is nice to foster some competition in hardware for model training, but the availability of these machines seems very limited - I don't think there's any major cloud provider allowing per hour rental of Gaudi2 VMs and Intel's own site directs you to buy an 8x GPU provisioned server from Supermicro for more than 40k USD. Availability and software stack is still heavily in Nvidia's favor right now, but maybe by the…
>Intel's own site directs you to buy an 8x GPU provisioned server from Supermicro for more than 40k USD
Isn't that the price of a single H100?
Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers
#18H100 was released almost exactly 1 year ago so I guess it's ok if Intel is now ready to compete with last year's model. To those commenting about "no moat" remember CUDA is a huge part of it, it's actually HW+SW and both took a decade to mature, together
It took less than a day to port our code over, we do custom CUDA across modalities.
Gaudi2 was actually announced 2 years ago and is 7nm like the A100 80Gb it was meant to be competitive with, Gaudi3 later this year is probably going to be the inflection point as that ramps
The cost is like 1/3
https://www.intel.com/content/www/us/en/newsroom/news/vision...
Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers
#19Hasn't H100 been shipping in volume for about a year already? Is Gaudi2 even available at comparable scale yet? I wouldn't count Nvidia out until they start slipping on similar timescales, i.e. if B100 doesn't have a clear lead over competing parts that become available at roughly the same time.
I think as we go to enterprise workloads the total cost of ownership becomes important.
NVIDIA is still the best for research given ecosystem but once the models are standardised as with transformers/LLaMA and likely multimodal diffusion transformers it then becomes about scale, availability and cost per flop.
Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers
#20Interesting! this was already the case with TPUs easily beating A100s. We sell Stable Diffusion finetuning on TPUs (dreamlook.ai), people are amazed how fast and cheap we can offer it - but there's no big secret, we just use hardware that's strictly faster and cheaper per unit of work. I expect a new wave of "your task, but on superior hardware" services to crop up with these chips!
Which TPUs do you use? Cloud-hosted or your own hardware? Interesting insight.