Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

tomshardware.com

Post reply on HN
Print view View original

301–310 of 380 posts

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#301

Post by why_only_15 »

ttoinou wrote: post #19
bartwr wrote: post #16

If they create a better tool chain, ecosystem, and programming experience than CUDA and compatible with all computational platforms at their peak performance - awesome! Everyone wins! Until then, it's a bit funny claim, especially considering what a failure OpenCL was (programmer's experience and fading support). Or trying to do GPGPU with compute shaders in DX/GL/Vulkan. Are they really "motivated"? Because they had…

What's wrong with compute shaders ?

Compute shaders are not capable of using modern GPU features like tensor cores or many of the other features needed to feed tensor cores data fast enough (e.g. TMA/cp.async.shared)

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#302

Post by soulbadguy »

jart wrote: post #283
shihab wrote: post #83

Earlier quoted context omitted.

But you can't support Pytorch without a proper foundation in place. They don't need to support zillion _end_ libraries, sure, but they do need to have at least a very good set of standard libraries, equivalent of Cublas, Curand etc. And they don't. My work recently had me working with rocRAND (Rocm's answer to Curand). It was frankly pretty bad- the design, performance (50% slower in places that don't make any sense…

I honestly don't see why it's so hard. On my project we wrote our own gemm kernels from scratch so llama.cpp didn't need to depend on cublas anymore. Only took a few days and a few hundred lines of code. We had to trade away 5% performance.

For a given set kernels, and a limited set of architectures, the problem is relatively easy.

But covering all the important kernels acros all the crazy architecture out there and with relatively good performance and numerical accuracy ... Much harder

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#304

Post by why_only_15 »

TheAlchemist wrote: post #149

Can anybody with a deep knowledge of the AI space, explain to me what's the real moat of CUDA ? It's clear to everybody that it's not the hardware but the software - which is the CUDA ecosystem. I've played a bit in the past with ML, but at the level of understanding I had - training some models, tweaking things, I was using higher level libraries and as far as I know, it's pretty much an if statement in those librar…

It is incorrect to say that the moat is the software. The moat is primarily the compute hardware, which is still incredibly good for the price, plus really good networking equipment. CUDA is not a significant moat for massive LLM training runs, as you can see from Anthropic moving from CUDA to Trainium (and so presumably rewriting all of their kernels to Trainium).

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#305

Post by rldjbpin »

also the market: let's all create our own CUDA.

this is where it is going all south imo. some initiatives push their "open-source" nature, but they are still distributed efforts in the end.

the vast majority of developers are using a framework and are completely abstracted out of the inner workings. they are not the main audience when marketting your ecosystem. but those are the ones calling the procurement shots unfortunately.

at this rate, some futuristic "Xeon Phi" like architecture, where the new surge of multithreaded processing on CPU side can just be generalized, might be more likely than "we got CUDA at home" nonsense.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#306

Post by rightbyte »

shihab wrote: post #83
lacker wrote: post #66

Earlier quoted context omitted.

The funny thing to me is that so much of the "AI software ecosystem" is just PyTorch. You don't need to develop some new framework and make it popular. You don't need to support a zillion end libraries. Just literally support PyTorch. If PyTorch worked fine on Intel GPUs, a lot of people would be happy to switch.

But you can't support Pytorch without a proper foundation in place. They don't need to support zillion _end_ libraries, sure, but they do need to have at least a very good set of standard libraries, equivalent of Cublas, Curand etc. And they don't. My work recently had me working with rocRAND (Rocm's answer to Curand). It was frankly pretty bad- the design, performance (50% slower in places that don't make any sense…

> generating random numbers

You can't bench implementations of random numbers against each other purely on execution speed.

A better algorithm (better statistical properties) will be slower.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#307

Post by Keyframe »

joe_the_user wrote: post #205
Keyframe wrote: post #78

Earlier quoted context omitted.

Guys need to do ye olde embrace, extend maneuver. What wine did, what Javas of the world did. CUDA driver API and CUDA runtime API either translation or implementation layer that offers compatibility and speed. I see no way around it at this point, for now.

They could do that. That would eliminate Nvidia's monopoly. AMD has made gestures in that direction with Hip. But they ultimately don't want to do that - Hip support is half-assed and inconsistent. AMD creates and abandons a variety of APIs. So the conclusion is the other chip makers whine about Nvidia's monopoly but don't want to end it - they just want maneuver to get their smaller monopolies of some sort or other.

You're right. At this stage CUDA is de facto what the standard is around. Just like in ISA wars x86 was. Doesn't matter if you have POWER whatever when everything's on the other thing. I get why not though, it would drag the battle onto Nvidia's home turf. At least it would be a battle though.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#308

Post by rightbyte »

goldenshale wrote: post #264
buildbot wrote: post #33

As another commenter said, it's CUDA. Intel and AMD and whoever can turn out chips reasonably fast, but nobody gets that it's the software and ecosystem. You have to out-compete the ecosystem. You can pick up a used Mi100 that performs almost like an A100 for 5x less money on eBay for example. Why is it 5x less? Because the software incompatibilities mean you'll spend a ton of time getting it to work compared to an N…

Seriously, why don't they just dedicate a group to creating the best pytorch backend possible? Proving it there will gain researcher traction and prove that their hardware is worth porting the other stuff over to.

You can't "just" do stuff like this. You need the right guy and big corps have no clue who is capable.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#309

Post by rightbyte »

ac29 wrote: post #187
brindlejim wrote: post #15

Fun fact: More than half of all engineers at NVIDIA are software engineers. Jensen has deliberately and strategically built a powerful software stack on top of his GPUs, and he's spent decades doing it. Until Intel finds a CEO who is as technical and strategic, as opposed to the bean-counters, I doubt that they will manage to organize a successful counterattack on CUDA.

Intel has over 15,000 software engineers, per their website. I couldn't find a number for NVIDIA, but it looks like they have a bit above 26k total employees. So, its very likely Intel has more software engineers than NVIDIA. Intel has far more products than NVIDIA though, so NVIDIA almost certainly has more software engineers working on GPU.

I would say that over a certain number of devs the output decreases.

I think it was 500 people working on Windows XP? A hundred for Windows 95. Etc.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#310

Post by vkazanov »

jauntywundrkind wrote: post #160
vkazanov wrote: post #131

Earlier quoted context omitted.

I used to play with these toys 7-8 years ago. We tried everything, and it was bad at it all. Traditional compute? The cores were too weak. Number crunching? Okay-ish but gpus were better. Useless stuff.

They seemed exceedingly hard to use well but interestingly capable & full of promise. And they were made in a much more primitive software age. I'd love to hear about what didn't work. OpenMP support seemed ok maybe but OpenMP is just a platform, figuring out software architectures that's mechanistically sympathetic to the system is hard. It would be so interesting to see what Xeon Phi might have been if we had Calci…

Well... Back then in my shop they would just assign programmers to things, together with a couple of mathematicians.

Math guys came up with a list of algorithms to try for a search engine backend.

What we needed was matrix multiplication and maybe some decision tree walking (that was some time ago, trees were still big back then, NNs were seen as too compute-intensive for no clear benefits). So we thought that it might be cool to have a tool that would support both. Phi sounded just right for both.

And things written to AVX-512 did work. Software surpisingly easy to port.

But then comes the usual SIMD/CPU trouble: every SIMD generation wants a little software rewrite. So for both Phi generations we had to update our code. For things not compatible with the SIMD approach (think tree-walking) it is just a weak x86.

In theory Phi's were universal, in practice what we got was: okay number crunching, bad generic compute.

GPU was somewhat similar: the software stack was unstable, CUDA just did not materialize as a standard yet. But every generation introduced a massive increase in compute available. And boy did NVIDIA move fast...

So GPU situation was: amazing number crunching, no generic compute.

And then there were a few ML breakthroughs results which rendered everything that did not look like a matrix multiplication obsolete.

PS I wouldn't take this story too seriously, details may vary.

Post reply on HN