Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

tomshardware.com

Post reply on HN
Print view View original

151–160 of 380 posts

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#151

Post by eternityforest »

singhrac wrote: post #103
shihab wrote: post #83

Earlier quoted context omitted.

But you can't support Pytorch without a proper foundation in place. They don't need to support zillion _end_ libraries, sure, but they do need to have at least a very good set of standard libraries, equivalent of Cublas, Curand etc. And they don't. My work recently had me working with rocRAND (Rocm's answer to Curand). It was frankly pretty bad- the design, performance (50% slower in places that don't make any sense…

Generating random numbers is a bit complicated! I wrote some of the samplers in Pytorch (probably replaced by now) and some of the underlying pseudo-random algorithms that work correctly in parallel are not exactly easy... running the same PRNG with the same seed on all your cores will produce the same result, which is probably NOT what you want from your API. But, to be honest, it's not that hard either. I'm surpris…

I wonder if the next generation chips are going to just have a dedicated hardware RNG per-core if that's an issue?

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#152

Post by selfhoster11 »

spacemanspiff01 wrote: post #114
singhrac wrote: post #106

Earlier quoted context omitted.

Just to point out it does, kind of: https://github.com/intel/intel-extension-for-pytorch I've asked before if they'll merge it back into PyTorch main and include it in the CI, not sure if they've done that yet. In this case I think the biggest bottleneck is just that they don't have a fast enough card that can compete with having a 3090 or an A100. And Gaudi is stuck on a different software platform which doesn't see…

They could compete on ram, if the software was there. Just having a low cost alternative to the 4060ti would allow them to break into the student/hobbies/open source market. I tried the a770, but returned it. Half the stuff does not work. They have the CPU side and GPU development on different branches (GPU seems to be ~6 months behind CPU) and often you have to compile it yourself, (if you want torchvision or torcha…

The amount of VRAM is the absolute killer USP for the current large AI model hobbyist segment. Something that had just as much VRAM as a 3090 but at half the speed and half the price would sell like hot cakes.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#153

Post by zozbot234 »

AnthonyMouse wrote: post #107
latchkey wrote: post #81

Earlier quoted context omitted.

People argue for ROCm to support older cards because that is all they have accessible to them. AMD has lagged on getting expensive cards into the hands of end users because they've focused only on building super computers. I predict that access to the newer cards is a more likely scenario. Right now, you can't rent a MI250 or even MI300x, but that is going to change quickly. Azure is going to have them, as well as ot…

> People argue for ROCm to support older cards because that is all they have accessible to them. What they really need is to support the less expensive cards, of which the older cards are a large subset. There are a lot of people who will make contributions and fix bugs if they can actually use the thing. Some CS student at the university has to pay tuition and therefore only has an old RX570, and that isn't going to…

The older cards are going to be supported by Mesa and RustiCL for the foreseeable future. Rocm is not the only game in town, far from it.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#154

Post by eternityforest »

lumost wrote: post #90
shihab wrote: post #83

Earlier quoted context omitted.

But you can't support Pytorch without a proper foundation in place. They don't need to support zillion _end_ libraries, sure, but they do need to have at least a very good set of standard libraries, equivalent of Cublas, Curand etc. And they don't. My work recently had me working with rocRAND (Rocm's answer to Curand). It was frankly pretty bad- the design, performance (50% slower in places that don't make any sense…

Folks also underestimate how complex these libraries are. There are dozens of projects to make BLAS alternatives which give up after ~3-6 months when they realize that this project will take years to be successful.

How does that work? Why not pick up where the previous team left off instead of everyone starting new ones? Or are they all targeting different backends and hardware?

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#155

Post by CamperBob2 »

_8j50 wrote: post #97

Intel's CEO should look into adversarial compatibility. The way you defeat a bigger/stronger enemy is not by attempting to be bigger and stronger than them but by using their strength (CUDA+great drivers) against them.

How would that play out, exactly?

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#156

Post by raphlinus »

mschuetz wrote: post #79
fluxem wrote: post #24

Earlier quoted context omitted.

CUDA = C++ on GPUs. Compute shader - subset of C with a weird quirks.

A subset that lacks pointers, which makes compute shaders a toy language next to CUDA.

Vulkan 1.3 has pointers, thanks to buffer device address[1]. It took a while to get there, and earlier pointer support was flawed. I also don't know of any major applications that use this.

Modern Vulkan is looking pretty good now. Cooperative matrix multiplication has also landed (as a widely supported extension), and I think it's fair to say it's gone past OpenCL.

Whether we get significant adoption of all this I think is too early to say, but I think it's a plausible foundation for real stuff. It's no longer just a toy.

[1] https://community.arm.com/arm-community-blogs/b/graphics-gam...

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#157

Post by raphlinus »

Const-me wrote: post #127
bartwr wrote: post #30

Earlier quoted context omitted.

I shipped a dozen products with them (mostly video games), so there's nothing "wrong" that would make them unusable. But programming them and setting up the graphics pipe (and all the passes, structured buffers, compiling, binding, weird errors, and synchronization) is a huge PITA as compared to the convenience of CUDA. Compilers are way less mature, especially on some platforms cough . Some GPU capabilities are not…

I have recently published an AI-related open-source project entirely based on compute shaders https://github.com/Const-me/Cgml and I’m super happy with the workflow. Possible to implement very complicated things without compiling a single line of C++, the software is mostly in C#. > setting up the graphics pipe I’ve picked D3D11, as opposed to D3D12 or Vulkan. The 11 is significantly higher level, and much easier to…

I understand why you've picked D3D11, but people have to understand that comes with serious limitations. There are no subgroups, which also means no cooperative matrix multiplication ("tensor cores"). For throughput in machine learning inference in particular, there's no way D3D11 can compete with either CUDA or a more modern compute shader stack, such as one based on Vulkan 1.3.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#158

Post by brindlejim »

tester756 wrote: post #125
brindlejim wrote: post #15

Fun fact: More than half of all engineers at NVIDIA are software engineers. Jensen has deliberately and strategically built a powerful software stack on top of his GPUs, and he's spent decades doing it. Until Intel finds a CEO who is as technical and strategic, as opposed to the bean-counters, I doubt that they will manage to organize a successful counterattack on CUDA.

>finds a CEO who is as technical and strategic, as opposed to the bean-counters Did you just call Gelsinger a "non-technical"? wow, how out of touch with reality >Gelsinger first joined Intel at 18 years old in 1979 just after earning an associate degree from Lincoln Tech.[9] He spent much of his career with the company in Oregon,[12] where he maintains a home.[13] In 1987, he co-authored his first book about program…

Intel spent more than a decade under Otellini, Krzanich and Swan. Bean counters. Gelsinger was appointed out of desperation, but the problem runs much deeper. I doubt that culture is gone. It has already cost Intel many opportunities.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#159

Post by haltist »

eternityforest wrote: post #154
lumost wrote: post #90

Earlier quoted context omitted.

Folks also underestimate how complex these libraries are. There are dozens of projects to make BLAS alternatives which give up after ~3-6 months when they realize that this project will take years to be successful.

How does that work? Why not pick up where the previous team left off instead of everyone starting new ones? Or are they all targeting different backends and hardware?

It's a compiler problem and there is no money in compilers [1]. If someone made an intermediate representation for AI graphs and then wrote a compiler from that intermediate format into whatever backend was the deployment target then they might be able to charge money for support and bug fixes but that would be it. It's not the kind of business anyone wants to be in so there is no good intermediate format and compiler that is platform agnostic.

1: https://tinygrad.org/

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#160

Post by jauntywundrkind »

vkazanov wrote: post #131
pjmlp wrote: post #117

Earlier quoted context omitted.

It was called Larrabee and XeonPhi, they botched it, and the only thing left from that effort is AVX.

I used to play with these toys 7-8 years ago. We tried everything, and it was bad at it all. Traditional compute? The cores were too weak. Number crunching? Okay-ish but gpus were better. Useless stuff.

They seemed exceedingly hard to use well but interestingly capable & full of promise. And they were made in a much more primitive software age.

I'd love to hear about what didn't work. OpenMP support seemed ok maybe but OpenMP is just a platform, figuring out software architectures that's mechanistically sympathetic to the system is hard. It would be so interesting to see what Xeon Phi might have been if we had Calcite or Velox or OpenXLA or other execution engine/optimizers that can orchestrate usage. The possibility of something like Phi seems so much higher now.

There's such a consensus around Phi tanking, and yes, some people came and tried and failed. But most of those lessons, of why it wasn't working (or was!) never survived the era, never were turned into stories & research that illuminates what Phi really was. My feeling is that most people were staying the course on GPU stuff, and that there weren't that many people trying Phi. I'd like more than the heresay heaped at Phi's feed to judge by.

Post reply on HN