Can Intel show us a GPU that beats Nvidia A100? Can they show us a framework for that GPU that is better than CUDA and which most companies are willing to adopt?
No? Then is just talk.
Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'
281–290 of 380 posts
Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'
#282Earlier quoted context omitted.
Open source generally wins once the state of the art has stopped moving. When a field is still experiencing rapid change closed source solutions generally do better than open source ones. Until we somehow figure out a relatively static set of requirements for running and training LLMs I wouldn’t expect any open source solution to win.
That doesn't really make sense. Pretty much all LLMs are trained in pytorch, which is open source. LLMs only reached the state it is now because many academic conferences insisted that paper submissions have open source code attached to it. So much of the ML/AI ecosystem is open source. Pretty much only CUDA is not open source.
>Pretty much only CUDA is not open source.
What stops Intel to make their own CUDA and plug in into pytorch?
Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'
#283Earlier quoted context omitted.
The funny thing to me is that so much of the "AI software ecosystem" is just PyTorch. You don't need to develop some new framework and make it popular. You don't need to support a zillion end libraries. Just literally support PyTorch. If PyTorch worked fine on Intel GPUs, a lot of people would be happy to switch.
But you can't support Pytorch without a proper foundation in place. They don't need to support zillion _end_ libraries, sure, but they do need to have at least a very good set of standard libraries, equivalent of Cublas, Curand etc. And they don't. My work recently had me working with rocRAND (Rocm's answer to Curand). It was frankly pretty bad- the design, performance (50% slower in places that don't make any sense…
I honestly don't see why it's so hard. On my project we wrote our own gemm kernels from scratch so llama.cpp didn't need to depend on cublas anymore. Only took a few days and a few hundred lines of code. We had to trade away 5% performance.
Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'
#284Earlier quoted context omitted.
> It's rare to find an operating system that doesn't implement the POSIX API. Except that it has barelly improved beyond CLI and daemons, still thinks terminals are the only hardware, everything else that matters isn't part of it, not even more modern networking protocols that aren't exposed in socket configurations.
Its purpose was to create compatibility between Unix vendors so developers could write software compatible with different flavors. The primary market for Unix vendors is servers, which to this day are still about CLI and daemons, and POSIX systems continue to have dominant market share in that market.
Maybe it's POSIX that is holding new developments back because people think it's good enough. It's not '60s anymore.
I would have expected to have totally new paradigms in 2023 if you asked me 23 years ago. Even NT kernel seems more modern.
While POSIX was state of the art when it was invented, it shouldn't be today.
Lots of research was thrown in recycle bin because "Hey, we have POSIX, why reinvent the wheel?", up to the point that nobody wants to do operating systems research today, because they don't want their hard work to get thrown into the same recycle bin.
I think that people who invented POSIX were innovators and have they live today, they would come with a totally new paradigm, more fit to today's needs and knowledge.
Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'
#285Earlier quoted context omitted.
That doesn't really make sense. Pretty much all LLMs are trained in pytorch, which is open source. LLMs only reached the state it is now because many academic conferences insisted that paper submissions have open source code attached to it. So much of the ML/AI ecosystem is open source. Pretty much only CUDA is not open source.
>Pretty much only CUDA is not open source. What stops Intel to make their own CUDA and plug in into pytorch?
The same shit as always, patents and copyright.
Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'
#286Earlier quoted context omitted.
This right here. Until Intel (and/or AMD) get serious about the software side and actually invest the money CUDA isn't going anywhere. Intel will make noises about various initiatives in that direction and then a quarter or two later they'll make big cuts in those divisions. They need to make a multi-year commitment and do some serious hiring (and they'll need to raise their salaries to market rates to do this) if th…
Literally, every single announcement (and action) coming out of AMD these days is that they are serious about the software. I don't see any reason at this point to doubt them. The larger issue is that they need to fix the access to their high end GPUs. You can't rent a MI250... or even a MI300x (yet, I'm working on that myself!). But that said, you can't rent an H100 either... there are none available.
>Literally, every single announcement (and action) coming out of AMD these days is that they are serious about the software. I don't see any reason at this point to doubt them.
I have a bridge to sell.
Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'
#287Earlier quoted context omitted.
Across the semi industry the prevailing attitude is that the chip is the important part and software is a nice add-on that you build as a one-off or that "the community" will support. This has changed much more slowly than I thought it would given NVIDIA's demonstration of what is possible when software is viewed as a core part of the product. Claiming the CUDA moat is weak suggests that it's business as usual over t…
Does the solution lie in genuinely investing in a CUDA alternative?
In my opinion the main issues are a layer up from that, it's more of a customer experience and brand building exercise to solve by actually improving the tools people use. Today NVIDIA is the default if you want to get work done and not have a lot of tooling issues. Nobody is going to forego that experience to work with flaky slow products from a late number three entrant but that's more or less the perception today. What's weird is they actually have the high ground in an important area: They ship more GPUs than just about everyone and have for a long time, their integrated graphics drivers are pretty good (virtualization!), and they have a deep bench of performance engineering talent. NVIDIA structurally has no way to reach the same volume. Intel with some investment of time could be the most productive platform for AI developers but it would take creativity that is hard in a big company and acknowledging that in that effort they're starting as a distant #3.
Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'
#288As another commenter said, it's CUDA. Intel and AMD and whoever can turn out chips reasonably fast, but nobody gets that it's the software and ecosystem. You have to out-compete the ecosystem. You can pick up a used Mi100 that performs almost like an A100 for 5x less money on eBay for example. Why is it 5x less? Because the software incompatibilities mean you'll spend a ton of time getting it to work compared to an N…
Both AMD and Intel (and Qualcomm to some degree) just don't seem to get how you beat NVIDIA. If they want to grab a piece of NVIDIA's pie, they do NOT need to build something better than an H100 right away. There are a million consumers who are happy with a 4090 or 4080 or even 3080 and would love for something that's equally capable at half price, and moreover, actually available for purchase, from Amazon/NewEgg/whe…
>Both AMD and Intel (and Qualcomm to some degree) just don't seem to get how you beat NVIDIA.
That's simple, just do what Nvidia did. Be better than the competition.
Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'
#289Earlier quoted context omitted.
Normally, I wouldn't do that, but this time I felt like both comments were intentionally inflammatory and then the context for my response was lost. "literally nothing" is also wrong given that they just had a large press announcement on Dec 6th (yt as part of my response below), where they spent 2 hours saying (and showing) they are serious. The second comment was made and then immediately deleted, in a way that was…
I deleted it because I was in an extremely bad mood and later realized it was simply wrong of me to post it and vent my unrelated frustration in those comments. I think it's in extremely bad taste to repost what I wrote when I made the clear choice to delete it.
If you didn't say it, the rocks would cry out. Personally, I'll believe AMD is maybe serious about Rocm if they make it a year without breaking their Debian repository.
Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'
#290Fun fact: More than half of all engineers at NVIDIA are software engineers. Jensen has deliberately and strategically built a powerful software stack on top of his GPUs, and he's spent decades doing it. Until Intel finds a CEO who is as technical and strategic, as opposed to the bean-counters, I doubt that they will manage to organize a successful counterattack on CUDA.
Ultimately it comes down to Intel and AMD being penny wise, pound foolish. They're unwilling to hire a lot of quality software engineers, because they're expensive, so they just continue to fall behind NVidia.