Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers

stability.ai

Post reply on HN
Print view View original

21–30 of 66 posts

Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers

#21

Post by ABS »

memossy wrote: post #18
ABS wrote: post #14

H100 was released almost exactly 1 year ago so I guess it's ok if Intel is now ready to compete with last year's model. To those commenting about "no moat" remember CUDA is a huge part of it, it's actually HW+SW and both took a decade to mature, together

It took less than a day to port our code over, we do custom CUDA across modalities. Gaudi2 was actually announced 2 years ago and is 7nm like the A100 80Gb it was meant to be competitive with, Gaudi3 later this year is probably going to be the inflection point as that ramps The cost is like 1/3 https://www.intel.com/content/www/us/en/newsroom/news/vision...

"Announced" 2 years ago is different from its availability and ability:

- Intel acquired Habana in 2019

- Habana launched Gaudi2 in 2022

- only in H2 2023 Habana enabled FP8 which delivered around 100% improvement in time-to-train

On the rest I believe you but markets don't move based on single individual's/company's data points

Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers

#22

Post by thunderbird120 »

Gaudi3 is supposedly due this year with a 4X bump in Bf16 training over Gaudi2. Gaudi is an interesting product. Intel seems to have something pretty decent but it hasn't seen much of a volume release yet. Maybe that comes with V3? Not sure exactly what their strategy with it is.

We do know that in 2025 it's supposed to be part of Intel's Falcon Shores HPC XPU. This essentially takes a whole bunch of HPC compute and sticks it all on the same silicon to maximize throughput and minimize latency. Thanks to their tile-based chip strategy they can have many different versions of the chip with different HPC focuses by swapping out different tiles. AI certainly seems to be a major one, but it will be interesting to see what products they come up with.

Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers

#23

Post by nabla9 »

memossy wrote: post #18
ABS wrote: post #14

H100 was released almost exactly 1 year ago so I guess it's ok if Intel is now ready to compete with last year's model. To those commenting about "no moat" remember CUDA is a huge part of it, it's actually HW+SW and both took a decade to mature, together

It took less than a day to port our code over, we do custom CUDA across modalities. Gaudi2 was actually announced 2 years ago and is 7nm like the A100 80Gb it was meant to be competitive with, Gaudi3 later this year is probably going to be the inflection point as that ramps The cost is like 1/3 https://www.intel.com/content/www/us/en/newsroom/news/vision...

  2024:  Nvidia's B100 TSMC 3nm (?)
  2024:  Intel Gaudi3  TSMC 5nm (*)
  2023:  AMD MI300X    TSMC 5nm/6nm 
  2022:  Nvidia H100   TSMC 4N
  2020   Nvidia A100   TSMC 7nm
(*): performance critical chiplets at least.

Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers

#24

Post by 1024core »

renewiltord wrote: post #20
MasterScrat wrote: post #4

Interesting! this was already the case with TPUs easily beating A100s. We sell Stable Diffusion finetuning on TPUs (dreamlook.ai), people are amazed how fast and cheap we can offer it - but there's no big secret, we just use hardware that's strictly faster and cheaper per unit of work. I expect a new wave of "your task, but on superior hardware" services to crop up with these chips!

Which TPUs do you use? Cloud-hosted or your own hardware? Interesting insight.

"TPUs" are a Google-only product, available* only on GCP.

* Notwithstanding the Choral boards

Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers

#28

Post by memossy »

renewiltord wrote: post #20
MasterScrat wrote: post #4

Interesting! this was already the case with TPUs easily beating A100s. We sell Stable Diffusion finetuning on TPUs (dreamlook.ai), people are amazed how fast and cheap we can offer it - but there's no big secret, we just use hardware that's strictly faster and cheaper per unit of work. I expect a new wave of "your task, but on superior hardware" services to crop up with these chips!

Which TPUs do you use? Cloud-hosted or your own hardware? Interesting insight.

We use v4s, v5es & v5ps. Mostly v5ps, very stable int8 training (versus the horror that is fp8 stability)

Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers

#29

Post by memossy »

ABS wrote: post #21
memossy wrote: post #18

Earlier quoted context omitted.

It took less than a day to port our code over, we do custom CUDA across modalities. Gaudi2 was actually announced 2 years ago and is 7nm like the A100 80Gb it was meant to be competitive with, Gaudi3 later this year is probably going to be the inflection point as that ramps The cost is like 1/3 https://www.intel.com/content/www/us/en/newsroom/news/vision...

"Announced" 2 years ago is different from its availability and ability: - Intel acquired Habana in 2019 - Habana launched Gaudi2 in 2022 - only in H2 2023 Habana enabled FP8 which delivered around 100% improvement in time-to-train On the rest I believe you but markets don't move based on single individual's/company's data points

Gaudi2s started coming out in 2022 (https://huggingface.co/blog/habana-gaudi-2-benchmark) but didn't hit mass scale. I think Gaudi3 will & others have seen similar performance for Gaudi2 eg Databricks: https://www.databricks.com/blog/llm-training-and-inference-i..., mlperf etc

We are about to drop stable diffusion 3 which is the best image model out there (https://x.com/EMostaque/status/1764941367682256950?s=20) with similar architecture to Sora by OpenAI that can be used for any modality.

We have hundreds of millions of downloads of our models so are looking for big scale as we move to every pixel being generated & this stuff goes from research to mass deployment.

Re: Intel Gaudi2 chips outperform Nvidia H100 on diffusion transformers

#30

Post by memossy »

nabla9 wrote: post #23
memossy wrote: post #18

Earlier quoted context omitted.

It took less than a day to port our code over, we do custom CUDA across modalities. Gaudi2 was actually announced 2 years ago and is 7nm like the A100 80Gb it was meant to be competitive with, Gaudi3 later this year is probably going to be the inflection point as that ramps The cost is like 1/3 https://www.intel.com/content/www/us/en/newsroom/news/vision...

2024: Nvidia's B100 TSMC 3nm (?) 2024: Intel Gaudi3 TSMC 5nm (*) 2023: AMD MI300X TSMC 5nm/6nm 2022: Nvidia H100 TSMC 4N 2020 Nvidia A100 TSMC 7nm (*): performance critical chiplets at least.

Falcon shores next year will be crazy with 300gb VRAM & new lith
Post reply on HN