Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

tomshardware.com

Post reply on HN
Print view View original

191–200 of 380 posts

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#191

Post by latchkey »

AnthonyMouse wrote: post #129
latchkey wrote: post #118

Earlier quoted context omitted.

What is nonsense is that you think that AMD should dedicate limited resources to supporting a 6 year old card with only 4-8gb of ram (the ones I ran had 8). I didn't say they are bad cards... they are just outdated at this point. If you really want to put your words to action... let me know. I'll put you in touch with someone to buy 130,000 of these cards, and you can sell them to every college kid out there... until…

8GB is enough for Stable Diffusion or Llama 13B q4. They're outdated, but non-outdated GPUs are still expensive , so they're all many people can afford. > I'll put you in touch with someone to buy 130,000 of these cards, and you can sell them to every college kid out there... Just sell them on eBay? They still go for $50-$100 each, so you're sitting on several million dollars worth of GPUs. > I'd personally rather se…

> Growing the community is how you catch up.

Agreed 100%.

> That doesn't happen if people can't afford the only GPUs you support.

On this part, we are going to have to agree to disagree. I feel like being able to at least affordably rent time on the high end GPUs is another alternative to buying them. As I mentioned above, that is something I'm actively working on.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#192

Post by latchkey »

lmm wrote: post #175
latchkey wrote: post #77

Earlier quoted context omitted.

Literally, every single announcement (and action) coming out of AMD these days is that they are serious about the software. I don't see any reason at this point to doubt them. The larger issue is that they need to fix the access to their high end GPUs. You can't rent a MI250... or even a MI300x (yet, I'm working on that myself!). But that said, you can't rent an H100 either... there are none available.

> Literally, every single announcement (and action) coming out of AMD these days is that they are serious about the software. I don't see any reason at this point to doubt them. They're having to announce it so much because people are rightly sceptical. Talk is cheap, and their software has sucked for years. Have they given concrete proof of their commitment, e.g. they've spent X dollars or hired Y people to work on…

Agreed. Time will tell.

MI300x and ROCm 6 and their support of projects like Pytorch, are all good steps in the right direction. HuggingFace now supports ROCm.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#193

Post by Const-me »

raphlinus wrote: post #180
Const-me wrote: post #169

Earlier quoted context omitted.

> no subgroups Indeed, in D3D they are called “wave intrinsics” and require D3D12. But that’s IMO a reasonable price to pay for hardware compatibility. > no cooperative matrix multiplication Matrix multiplication compute shader which uses group shared memory for cooperative loads: https://github.com/Const-me/Cgml/blob/master/Mistral/Mistral... > tensor cores When running inference on end-user computers, for many prac…

Sure. It's a tradeoff space. Gain portability and ergonomics, lose throughput. For applications that are throttled by TOPS at low precisions (ie most ML inferencing) then the performance drop from not being able to use tensor cores is going to be unacceptable. Glad you found something that works for you, but it certainly doesn't spell the end of CUDA.

> ie most ML inferencing

Most ML inferencing is throttled with memory, not compute. This certainly applies to both Whisper and Mistral models.

> it certainly doesn't spell the end of CUDA

No, because traditional HPC. Some people in the industry spent many man-years developing very complicated compute kernels, which are very expensive to port.

AI is another story. Not too hard to port from CUDA to compute shaders, because the GPU-running code is rather simple.

Moreover, it can help with performance just by removing abstraction layers. I think the reason why compute shaders-based Whisper outperformed CUDA-based version on the same GPU, these implementations do slightly different things. Unlike Python and Torch, compute shaders actually program GPUs as opposed to calling libraries with tons of abstractions layers inside them. This saves memory bandwidth storing and then loading temporary tensors.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#194

Post by tormeh »

AnthonyMouse wrote: post #96
foobiekr wrote: post #64

Earlier quoted context omitted.

It's not just Intel. Open initiatives and consortiums (the phase two of the same) are always the losers ganging up hoping that it will give them the leg up they don't have. If you're older you'll have seen this play out over and over in the industry - the history of Unix vs. Windows NT from the 1990s was full of actions like this, networking is going through it again for the nth time (this time with UltraEthernet) an…

You're just listing the ones that didn't knock it out of the park. TCP/IP completely displaced IPX to the point that most people don't even remember what it was. Nobody uses WINS anymore, even Microsoft uses DNS. It's rare to find an operating system that doesn't implement the POSIX API. The past is littered with the corpses of proprietary technologies displaced by open standards. Because customers don't actually wan…

Open source generally wins once the state of the art has stopped moving. When a field is still experiencing rapid change closed source solutions generally do better than open source ones. Until we somehow figure out a relatively static set of requirements for running and training LLMs I wouldn’t expect any open source solution to win.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#195

Post by [deleted] »

signatoremo wrote: post #136
ActionHank wrote: post #76

Earlier quoted context omitted.

We're in 2023, he's been in the CEO seat for 2 years already. He's had plenty of time to show the world his intent and where they are going. All that has happened is they launched a very mid GPU and have yielded more ground to AMD. Meanwhile AMD continue to eat away at Intel's talent pool, market share, and still managed to push into the AI space. He should be sweating.

His intent is “5 nodes in 4 years” - [0]. The goal is to reclaim the node leadership from TSMC by 2025. They announced the first chips based on Intel 4 today, which is more or less equivalent to TSMC’s 5nm. They may fail, but the goal is clear and ambitious. [0] - https://www.xda-developers.com/intel-roadmap-2025-explainer/

[deleted]

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#196

Post by eyegor »

selfhoster11 wrote: post #152
spacemanspiff01 wrote: post #114

Earlier quoted context omitted.

They could compete on ram, if the software was there. Just having a low cost alternative to the 4060ti would allow them to break into the student/hobbies/open source market. I tried the a770, but returned it. Half the stuff does not work. They have the CPU side and GPU development on different branches (GPU seems to be ~6 months behind CPU) and often you have to compile it yourself, (if you want torchvision or torcha…

The amount of VRAM is the absolute killer USP for the current large AI model hobbyist segment. Something that had just as much VRAM as a 3090 but at half the speed and half the price would sell like hot cakes.

You are describing the ebay market for used nvidia tesla cards. The k80, p40, or m40 are widely available and sell for ~$100 with 24gb vram. The m10 even has 32gb! The problem for ai hobbyists is it won't take long to realize how many apis use the "optical flow" pathways and so on nvidia they'll only run at acceptable speeds on rtx hardware, assuming they run at all. Cuda versions are pinned to hardware to some extent.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#197

Post by aseipp »

ac29 wrote: post #190
aseipp wrote: post #182

Earlier quoted context omitted.

It doesn't work if you're going against GPUs. All the nice goodies we are accustomed to on large desktop x86 machines with gigantic caches and huge branch predictor area and OOO execution engines -- the features that yield the performance profile we expect -- simply do not translate or scale up to thousands of cores per die. To scale that up, you need to redesign the microarchitecture in a fundamental way to allow mo…

While the first >1000 core x86 processor is probably a little ways out, Intel is releasing a 288-core x86 processor in the first half of 2024 (Sierra Forest). I assume AMD will have something similarly high core in 2024-25 as well.

To be clear, you can probably make a 1000 core x86 machine, and those 1000 cores can probably even be pretty powerful. I don't doubt that. I think Azure even has crazy 8-socket multi-sled systems doing hundreds of cores, today. But this thread is about CUDA. Sierra Forest will get absolutely obliterated by a single A100 in basically any workload where you could reasonably choose between the two as options. I'm not saying they can't exist. Just that they will be (very) bad in this specific competition. I made an edit to my comment to reflect that.

But what you mention is important, and also a reason for the ultimate demise of e.g. Xeon Phi. Intel surely realized they could just scale their existing Xeon designs up-and-out further than expected. Like from a product/SKU standpoint, what is the point of having a 300 core Phi where every core is slow as shit, when you have a 100 core 4-socket Xeon design on the horizon, using an existing battle-tested design that you ship billions of dollars worth every year? Especially when the 300 core Xeon fails completely against the competition. By the time Phi died, they were already doing 100-cores-per-socket systems. They essentially realized any market they could have had would be served better by the existing Xeon line and by playing to their existing strengths.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#198

Post by raphlinus »

zozbot234 wrote: post #186
raphlinus wrote: post #178

Earlier quoted context omitted.

That's a really good question. I don't know enough about SYCL to be able to tell you the answer, but I've heard rumblings that it may be the thing to watch. I think there may be some other limitations, for example SYCL 2020 depends on unified shared memory, and that is definitely not something you can depend on in compute shader land (in some cases you can get some of it, for example with resizable BAR, but it depend…

https://enccs.github.io/sycl-workshop/unified-shared-memory/ seems to suggest that USM is still a hardware-specific feature in SYCL 2020, so compatibility with hardware that requires a buffer copying approach is still maintained. Is this incorrect?

Good call. So this doesn't look like a blocker to SYCL compatibility. I'm interested in learning more about this.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#199

Post by carterschonwald »

shihab wrote: post #173
singhrac wrote: post #103

Earlier quoted context omitted.

Generating random numbers is a bit complicated! I wrote some of the samplers in Pytorch (probably replaced by now) and some of the underlying pseudo-random algorithms that work correctly in parallel are not exactly easy... running the same PRNG with the same seed on all your cores will produce the same result, which is probably NOT what you want from your API. But, to be honest, it's not that hard either. I'm surpris…

> Generating random numbers is a bit complicated! I know! I just wrote a whole paper and published a library on this! But really, perhaps not as much as many from outside might think. The core of a Philox implementation can be around 50 lines of C++ [1], with all the bells and whistles maybe around 300-400. That implementation's performance equals CuRAND's , sometimes even surpasses it! (the API is designed to avoid…

Cool! I’ll have a lookseee. I’ve my own experiments in this space.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#200

Post by natbennett »

AnthonyMouse wrote: post #142
nvm0n2 wrote: post #137

Earlier quoted context omitted.

Arguably the dominant APIs in the server space are the cloud APIs not POSIX.

Many of which are also open, like OpenStack or K8s, or have third party implementations, like Ceph implementing the Amazon S3 API.

Also all reimplementations of proprietary technology.

The S3 API is a really good example of the “OSS only becomes dominant when development slows down” principle. As a friend of mine who has had to support a lot of local blob storage says, “On the gates of hell are emblazoned — S3 compatible.”

Post reply on HN