Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

tomshardware.com

Post reply on HN
Print view View original

321–330 of 380 posts

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#321

Post by overdrive23 »

epolanski wrote: post #314
seanalltogether wrote: post #299

Earlier quoted context omitted.

Nvidia GPU moat has always been their software. Game ready drivers are a big deal for each AAA game launch and they always help to push their fps numbers on reviewers charts. I feel like for 20 years I've been reading people online complain about ATI/AMD drivers and how they want to go back to an Nvidia card the next chance they get.

This hasn't been true for more than a decade at this point, and in fact AMD tends to have the better driver support, especially long term.

Well, you said a decade so I'll take an easy one from 4 years ago.

"Yes, you know all those r/amd and r/nvidia posts about people ditching their RX 5700 XT and switching over to an Nvidia RTX 2070 Super… AMD is reading them and has obviously been jamming them down the throats of its software engineers until they could release a driver patch which addresses the issues Navi users have been experiencing."

https://www.pcgamesn.com/amd/radeon-rx-5700-drivers-black-sc...

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#323

Post by imtringued »

why_only_15 wrote: post #300
imtringued wrote: post #298

Earlier quoted context omitted.

You could do sparse deep learning with much, much larger models with these CPUs. As paradoxical as it might sound, sparse deep learning gets more compute bound as you add more cores.

I'd be curious to learn more about how it's compute bound and what specifically is compute bound. On modern H100s you need ~600 fp8 operations per byte loaded from memory in order to be compute bound, and that's with full 128-byte loads each time. Even integer/fp32 vector operations need quite a few operations to be compute bound (~20 for vector fp32).

I think you misunderstood what I mean. Sparse ML is inherently memory latency bound since you have a completely unpredictable access pattern prone to cache misses. The amount of compute you perform is a tiny blip compared to the hash map operations you perform. What I mean is that as you add more cores, there are sharing effects because multiple cores are accessing the same memory location at the same time. The compute bound sections of your code become a much greater percentage of the overall runtime as you add cores, which is surprising, since adding more compute is the easy part. Pay attention to my words "_more_ compute bound".

Here is a relevant article: https://www.kdnuggets.com/2020/03/deep-learning-breakthrough...

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#324

Post by donor20 »

jillesvangurp wrote: post #296
tester756 wrote: post #125

Earlier quoted context omitted.

>finds a CEO who is as technical and strategic, as opposed to the bean-counters Did you just call Gelsinger a "non-technical"? wow, how out of touch with reality >Gelsinger first joined Intel at 18 years old in 1979 just after earning an associate degree from Lincoln Tech.[9] He spent much of his career with the company in Oregon,[12] where he maintains a home.[13] In 1987, he co-authored his first book about program…

Gelsinger is a typical hardware engineer out of his depth competing against what is effctively a software play. This is a recurring theme in the industry where you have successful hardware companies with strong hardware focused leadership fail over time because they don't get software. I used to work at Nokia Research. The problem was on full display during the period Apple made it's entry into mobile. We had plenty…

apple did initially want to build on what they saw as the best fab - Intel. unlock the power Intel would bring for their phone. But they had some design objectives focused on user experience (power/cost) and Intel didn't see the value. Intel then scrambled to try and build what apple had asked for but without the software.

Nokia kept doing crazy hardware to show off on hardware side. But these old companies can't stop nickle and dimeing - so you'd get stuff w crazy drm etc. And software wasn't there or invested in fully.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#325

Post by muxr »

bragr wrote: post #34

People don't seem particularly motivated to move away from CUDA to me. I've been poking around various models and tooling over the last couple months, and they pretty much all have something like device = "cuda" if torch.cuda.is_available() else "cpu" and I've yet to see a single one implement the AMD NN middleware: https://www.amd.com/en/developer/zendnn.html

zendnn is for CPUs. Pytorch ROCm version doesn't require you to change a single line of code. So all the same semantics like `device = "cuda"` work the same when using AMD GPUs.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#326

Post by flkenosad »

amadeuspagel wrote: post #91
roenxi wrote: post #55

Earlier quoted context omitted.

Gelsinger is saying "the entire industry" and that seems likely to be a simple fact. Every single player, other than Nvidia, has an incentive to minimise the importance of CUDA as a proprietary technology. That is a lot more programmers than Nvidia can afford to employ. Even if Intel falls over its own feet, the incentives to bring in more chip manufacturers are huge. It'll happen, the only question is whether the ti…

> That is a lot more programmers than Nvidia can afford to employ. How many programmers one can employ is determined by profits, and Nvidia has monopoly profits thanks to CUDA, while "the entire industry" can at best hope to create some commiditized alternative to CUDA. Companies with real market power can beat entire industries of commodity manufacturers, Apple is the prime example.

How many programmers do you really need though to catch up to what CUDA has already? The path has been laid. There's no need for experimentation. Just copy what NVIDIA did. No?

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#327

Post by treprinum »

dheera wrote: post #170
buildbot wrote: post #33

As another commenter said, it's CUDA. Intel and AMD and whoever can turn out chips reasonably fast, but nobody gets that it's the software and ecosystem. You have to out-compete the ecosystem. You can pick up a used Mi100 that performs almost like an A100 for 5x less money on eBay for example. Why is it 5x less? Because the software incompatibilities mean you'll spend a ton of time getting it to work compared to an N…

Both AMD and Intel (and Qualcomm to some degree) just don't seem to get how you beat NVIDIA. If they want to grab a piece of NVIDIA's pie, they do NOT need to build something better than an H100 right away. There are a million consumers who are happy with a 4090 or 4080 or even 3080 and would love for something that's equally capable at half price, and moreover, actually available for purchase, from Amazon/NewEgg/whe…

That's not going to work because each GPU has a different internal architecture and aligning the way data is fed with how stream processors operate is different for each architecture (stuff like how memory buffers are organized/aligned/paged etc.). AMD is very incompatible to Nvidia at the lowest level so things/approaches that are fast on Nvidia can be 10x slower on AMD and vice versa.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#328

Post by muxr »

icelancer wrote: post #211
eyegor wrote: post #196

Earlier quoted context omitted.

You are describing the ebay market for used nvidia tesla cards. The k80, p40, or m40 are widely available and sell for ~$100 with 24gb vram. The m10 even has 32gb! The problem for ai hobbyists is it won't take long to realize how many apis use the "optical flow" pathways and so on nvidia they'll only run at acceptable speeds on rtx hardware, assuming they run at all. Cuda versions are pinned to hardware to some exten…

Yep. I have a fleet of P40s that are good at what they do (Whisper ASR primarily) but anything even remotely new... nah. fp16 support is missing so you need P100 cards, and usually that means you are accepting 16GB of VRAM rather than 24GB. Still some cool hardware.

For us hobbyists used 3090 or new 7900xtx seem to be the way. But even then you still need to build a machine with 3 or 4 of these GPUs to get enough VRAM to play with big models.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#329

Post by muxr »

brindlejim wrote: post #15

Fun fact: More than half of all engineers at NVIDIA are software engineers. Jensen has deliberately and strategically built a powerful software stack on top of his GPUs, and he's spent decades doing it. Until Intel finds a CEO who is as technical and strategic, as opposed to the bean-counters, I doubt that they will manage to organize a successful counterattack on CUDA.

People used the same argument when saying AMD would never beat Intel in CPUs. Intel has a lot of software engineers. Also these days AMD has a good number of software folks, thanks to the Xilinx acquisition and the organic investments in this area.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#330

Post by lelanthran »

eternityforest wrote: post #151
singhrac wrote: post #103

Earlier quoted context omitted.

Generating random numbers is a bit complicated! I wrote some of the samplers in Pytorch (probably replaced by now) and some of the underlying pseudo-random algorithms that work correctly in parallel are not exactly easy... running the same PRNG with the same seed on all your cores will produce the same result, which is probably NOT what you want from your API. But, to be honest, it's not that hard either. I'm surpris…

I wonder if the next generation chips are going to just have a dedicated hardware RNG per-core if that's an issue?

Why bother?

It's not the generation that matters so much, it's the gathering of entropy, which comes from peripherals and not possible to generate on-die.

If you don't need cryptographically secure randomness, you still want the entropy for generating the seeds per thread/die/chip.

Post reply on HN