Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

tomshardware.com

Post reply on HN
Print view View original

291–300 of 380 posts

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#291

Post by slavik81 »

flamedoge wrote: post #263
roenxi wrote: post #214

Earlier quoted context omitted.

I suppose, but it is a practical matter here. CUDA is a library for memory management and matrix math targeted at researchers, hyper-productive devs and enthusiasts. It looks like it'll be highly capital intensive, requiring hardware that runs in some of the biggest, nastiest, OSS-friendliest data-centres in the world who all design their own silicon. The generations of AMD GPU that matter - the ones out and on peopl…

AMD could use testers(cough clients i mean) like you. Jokes aside, please report bugs to rocm github..

Unless their hardware is on the official support list, I wouldn't be too hopeful for a quick resolution. Still, it's even less likely to get fixed if it's not reported.

If nothing else, I would be curious to know more about the issue. Personally, I want to know how well ROCm functions on every AMD GPU.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#292

Post by nemothekid »

DeathArrow wrote: post #282
kelipso wrote: post #213

Earlier quoted context omitted.

That doesn't really make sense. Pretty much all LLMs are trained in pytorch, which is open source. LLMs only reached the state it is now because many academic conferences insisted that paper submissions have open source code attached to it. So much of the ML/AI ecosystem is open source. Pretty much only CUDA is not open source.

>Pretty much only CUDA is not open source. What stops Intel to make their own CUDA and plug in into pytorch?

CUDA is huge and nvidia spent a ton in a lot of "dead end" use cases optimizing it. There have been experiments with CUDA translation layers with decent performance[1]. There are two things that most projects hit:

1. The CUDA API is huge; I'm sure Intel/AMD will focus on what they need to implement pytorch and ignore every other use case ensuring that CUDA always has the leg up in any new frontier

2. Nvidia actually cares about developer experience. The most prominent example is Geohotz with tinygrad - where AMD examples didn't even work or had glaring compiler bugs. You will find nvidia engineer in github issues for CUDA projects. Intel/AMD hasn't made that level of investment and thats important because GPUs tend to be more fickle than CPUs.

[1] https://github.com/vosen/ZLUDA

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#293

Post by MrBuddyCasino »

buildbot wrote: post #33

As another commenter said, it's CUDA. Intel and AMD and whoever can turn out chips reasonably fast, but nobody gets that it's the software and ecosystem. You have to out-compete the ecosystem. You can pick up a used Mi100 that performs almost like an A100 for 5x less money on eBay for example. Why is it 5x less? Because the software incompatibilities mean you'll spend a ton of time getting it to work compared to an N…

They still don't get that those who are serious about hardware, must make their own software.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#294

Post by selfhoster11 »

eyegor wrote: post #196
selfhoster11 wrote: post #152

Earlier quoted context omitted.

The amount of VRAM is the absolute killer USP for the current large AI model hobbyist segment. Something that had just as much VRAM as a 3090 but at half the speed and half the price would sell like hot cakes.

You are describing the ebay market for used nvidia tesla cards. The k80, p40, or m40 are widely available and sell for ~$100 with 24gb vram. The m10 even has 32gb! The problem for ai hobbyists is it won't take long to realize how many apis use the "optical flow" pathways and so on nvidia they'll only run at acceptable speeds on rtx hardware, assuming they run at all. Cuda versions are pinned to hardware to some exten…

Out of these three, only P40 is worth the effort to get running vs the capabilities they offer. That's also before considering that other than software hacks or configuration tweaks, those cards require specialised cooling shrouds for adequate cooling in tower-style cases.

If your time or personal energy is worth >$0, these cards work out to much more than $100. And you can't even file the time burnt on getting them to run as any kind of transferable experience.

That's not to say I don't recommend getting them - I have a P4 family card and will get at least one more, but I'm not kidding myself that the use isn't very limited.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#295

Post by 65a »

DeathArrow wrote: post #286
latchkey wrote: post #77

Earlier quoted context omitted.

Literally, every single announcement (and action) coming out of AMD these days is that they are serious about the software. I don't see any reason at this point to doubt them. The larger issue is that they need to fix the access to their high end GPUs. You can't rent a MI250... or even a MI300x (yet, I'm working on that myself!). But that said, you can't rent an H100 either... there are none available.

>Literally, every single announcement (and action) coming out of AMD these days is that they are serious about the software. I don't see any reason at this point to doubt them. I have a bridge to sell.

It is way, way better in the last year or so. Perfectly reasonable cards for inference if you actually understand the stack and know how to use it. Is nVidia faster? Sure, but at twice the price for 20-30% gains. If that makes sense for you, keep paying the tax.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#296

Post by jillesvangurp »

tester756 wrote: post #125
brindlejim wrote: post #15

Fun fact: More than half of all engineers at NVIDIA are software engineers. Jensen has deliberately and strategically built a powerful software stack on top of his GPUs, and he's spent decades doing it. Until Intel finds a CEO who is as technical and strategic, as opposed to the bean-counters, I doubt that they will manage to organize a successful counterattack on CUDA.

>finds a CEO who is as technical and strategic, as opposed to the bean-counters Did you just call Gelsinger a "non-technical"? wow, how out of touch with reality >Gelsinger first joined Intel at 18 years old in 1979 just after earning an associate degree from Lincoln Tech.[9] He spent much of his career with the company in Oregon,[12] where he maintains a home.[13] In 1987, he co-authored his first book about program…

Gelsinger is a typical hardware engineer out of his depth competing against what is effctively a software play. This is a recurring theme in the industry where you have successful hardware companies with strong hardware focused leadership fail over time because they don't get software.

I used to work at Nokia Research. The problem was on full display during the period Apple made it's entry into mobile. We had plenty of great software people throughout the company. But the leadership had grown up in a world where Nokia was basically making and selling hardware. Radio engineers and hardware engineers basically. They did not get software all that well. And of course what Apple did was executing really well on software for what was initially a nice but not particularly impressive bit of hardware. It's the software that made the difference. The hardware excellence came later. And the software only got better over time. Nokia never recovered from that. And they tried really hard to fix the software. It failed. They couldn't do it. Symbian was a train wreck and flopped hard in the market.

Intel is facing the same issue here. Their hardware is only useful if there's great software to do something with it. The whole point of hardware is running software. And Intel is not in the software business so they need others to do that for them. Similar to Nokia, Apple came along and showed the world that you don't need Intel hardware to deliver a great software experience. Now their competitor NVidia is basically stealing their thunder in the AI and 3D graphics market. Intel wants in but just like they failed to get into the mobile market (they tried, with Nokia even), their efforts to enter this market are also crippled by their software ineptness.

This is a lesson that many IOT companies struggle with as well. Great hardware but they typically struggle with their software ecosystems and unlocking the value of the hardware. So much so that one Finnish software company in this space (Wirepas), has been running an absolute genius marketing campaign with the beautiful slogan "Most IOT is shit". Check out their website. Some very nice Finnish humor on display there. Their blunt message is that most hardware focused IOT companies are hopelessly clumsy on the software front and they of course provide a solution.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#297

Post by nox100 »

buildbot wrote: post #33

As another commenter said, it's CUDA. Intel and AMD and whoever can turn out chips reasonably fast, but nobody gets that it's the software and ecosystem. You have to out-compete the ecosystem. You can pick up a used Mi100 that performs almost like an A100 for 5x less money on eBay for example. Why is it 5x less? Because the software incompatibilities mean you'll spend a ton of time getting it to work compared to an N…

Is Mojo trying to solve this?

https://www.youtube.com/watch?v=SEwTjZvy8vw

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#298

Post by imtringued »

dahart wrote: post #257
ac29 wrote: post #190

Earlier quoted context omitted.

While the first >1000 core x86 processor is probably a little ways out, Intel is releasing a 288-core x86 processor in the first half of 2024 (Sierra Forest). I assume AMD will have something similarly high core in 2024-25 as well.

> Intel is releasing a 288-core x86 This made me wonder a couple of things- What kind of workloads and problems is that best suited for? It’s a lot of cores for a CPU, but for pure math/compute, like with AI training and inference and with graphics, 288 cores is like ~1.5% of the number of threads of a modern GPU, right? Doesn’t it take particular kinds of problems to make a 288 core CPU attractive? I also wondered i…

You could do sparse deep learning with much, much larger models with these CPUs. As paradoxical as it might sound, sparse deep learning gets more compute bound as you add more cores.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#299

Post by seanalltogether »

brindlejim wrote: post #15

Fun fact: More than half of all engineers at NVIDIA are software engineers. Jensen has deliberately and strategically built a powerful software stack on top of his GPUs, and he's spent decades doing it. Until Intel finds a CEO who is as technical and strategic, as opposed to the bean-counters, I doubt that they will manage to organize a successful counterattack on CUDA.

Nvidia GPU moat has always been their software. Game ready drivers are a big deal for each AAA game launch and they always help to push their fps numbers on reviewers charts. I feel like for 20 years I've been reading people online complain about ATI/AMD drivers and how they want to go back to an Nvidia card the next chance they get.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#300

Post by why_only_15 »

imtringued wrote: post #298
dahart wrote: post #257

Earlier quoted context omitted.

> Intel is releasing a 288-core x86 This made me wonder a couple of things- What kind of workloads and problems is that best suited for? It’s a lot of cores for a CPU, but for pure math/compute, like with AI training and inference and with graphics, 288 cores is like ~1.5% of the number of threads of a modern GPU, right? Doesn’t it take particular kinds of problems to make a 288 core CPU attractive? I also wondered i…

You could do sparse deep learning with much, much larger models with these CPUs. As paradoxical as it might sound, sparse deep learning gets more compute bound as you add more cores.

I'd be curious to learn more about how it's compute bound and what specifically is compute bound. On modern H100s you need ~600 fp8 operations per byte loaded from memory in order to be compute bound, and that's with full 128-byte loads each time. Even integer/fp32 vector operations need quite a few operations to be compute bound (~20 for vector fp32).
Post reply on HN