Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

tomshardware.com

Post reply on HN
Print view View original

341–350 of 380 posts

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#341

Post by CamperBob2 »

_8j50 wrote: post #269
anon291 wrote: post #259

Earlier quoted context omitted.

Cuds api and language are easy. Nvidia has dozens of cuda offshoot projects that are harder.

If it's easy why don't they support it instead of watch billions of dollars fly away I wonder. If whatever cuda program you build would run the same on intel or amd, do those offshoot projects really matter?

I don't get either, to be honest. This isn't Direct3D versus OpenGL Part Deux, where a few different companies are vying for the approval of some game-tech geeks as a side business, it's the future of computing. If you make CPUs or GPUs and your building doesn't have a sign reading "nVidia" out front, it's currently on fire. You should probably be doing something about that besides talking about how motivated you are.

Nobody really seems to be taking CUDA compatibility seriously.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#342

Post by madaxe_again »

tw04 wrote: post #333
madaxe_again wrote: post #276

Earlier quoted context omitted.

Which is why intel left nvidia and everyone else in the dust with CUDA, which they developed. Oh, wait…

What exactly do you think Intel was going to write CUDA for? Their GPU products have been on the market less than 2 years, and they're still trying to get their arms wrapped around drivers. Them understanding what was coming doesn't mean they have a magic wand to instantly have a fully competitive product. You can't write a CUDA competitor until you've gotten the framework laid. The fact they invested so heavily in t…

That’s my point though - they’re catching up, not leading, which rather implies that they absolutely missed a beat, and don’t therefore understand where the market is going before it goes there.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#343

Post by zozbot234 »

PeterisP wrote: post #336
zozbot234 wrote: post #153

Earlier quoted context omitted.

The older cards are going to be supported by Mesa and RustiCL for the foreseeable future. Rocm is not the only game in town, far from it.

Thing is, for there to be a realistic alternative to CUDA, something needs to become "the only game in town" because people definitely won't add support for ROCm and Mesa and RustiCL and something else; getting support for one non-CUDA thing already is proving to be too difficult, so if the alternatives are fragmented, that makes the situation even worse.

RustiCL is just an OpenCL implementation. You can't have one-size-fits-all because hardware-specific stuff varies a lot across generations. (Which is also why you have newer versions of e.g. Rocm dropping support for older hardware.) The best you can do is have baseline support + extensions, which is the Vulkan approach.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#344

Post by refulgentis »

ilaksh wrote: post #273
refulgentis wrote: post #233

Earlier quoted context omitted.

People have really bizarre overdramatic misunderstandings of llama.cpp because they used it a few times to cook their laptop. This one really got me giggling though.

I am integrating llama.cpp into my application. I just went through one of their text generation examples line-by-line and converted it into my own class. This is a leading-edge software library that provides a huge boost for non-Nvidia hardware in terms of inference capability with quantized models. If you don't understand that, then you have missed an important development in the space of machine learning.

Jeez. Lol.

At length:

- yes, local inference is good. I can't say this strongly enough: llama.cpp is a fraction of a fraction of local inference.

- avoid talking down to people and histrionics. It's a hot field, you're in it, but like all of us always, you're still learning. When faced with a contradiction, check your premises, then share them.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#345

Post by eternityforest »

lelanthran wrote: post #330
eternityforest wrote: post #151

Earlier quoted context omitted.

I wonder if the next generation chips are going to just have a dedicated hardware RNG per-core if that's an issue?

Why bother? It's not the generation that matters so much, it's the gathering of entropy, which comes from peripherals and not possible to generate on-die. If you don't need cryptographically secure randomness, you still want the entropy for generating the seeds per thread/die/chip.

It absolutely is possible to generate entropy on-die, assuming you actually want entropy and not just a unique value that gets XORed with the seed, so you can still have repeatable seeds.

Pretty much every chip has an RNG which can be as simple as just a single free running oscillator you sample

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#347

Post by icelancer »

muxr wrote: post #328
icelancer wrote: post #211

Earlier quoted context omitted.

Yep. I have a fleet of P40s that are good at what they do (Whisper ASR primarily) but anything even remotely new... nah. fp16 support is missing so you need P100 cards, and usually that means you are accepting 16GB of VRAM rather than 24GB. Still some cool hardware.

For us hobbyists used 3090 or new 7900xtx seem to be the way. But even then you still need to build a machine with 3 or 4 of these GPUs to get enough VRAM to play with big models.

For sure - our prod machine has 6x RTX 3090s on some old cirrascale hardware. But P40s are still good for last-gen models. Just nothing new unfortunately.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#348

Post by icelancer »

selfhoster11 wrote: post #294
eyegor wrote: post #196

Earlier quoted context omitted.

You are describing the ebay market for used nvidia tesla cards. The k80, p40, or m40 are widely available and sell for ~$100 with 24gb vram. The m10 even has 32gb! The problem for ai hobbyists is it won't take long to realize how many apis use the "optical flow" pathways and so on nvidia they'll only run at acceptable speeds on rtx hardware, assuming they run at all. Cuda versions are pinned to hardware to some exten…

Out of these three, only P40 is worth the effort to get running vs the capabilities they offer. That's also before considering that other than software hacks or configuration tweaks, those cards require specialised cooling shrouds for adequate cooling in tower-style cases. If your time or personal energy is worth >$0, these cards work out to much more than $100. And you can't even file the time burnt on getting them…

The K80 is literally worth P40s are definitely the best value along with P100s. I still think there is a good amount of value to be extracted from both of those cards, especially if you are interested in using Whisper ASR, video transcoding, or CUDA models that were relevant before LLMs (a time many people have forgotten apparently).

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#349

Post by shihab »

rightbyte wrote: post #306
shihab wrote: post #83

Earlier quoted context omitted.

But you can't support Pytorch without a proper foundation in place. They don't need to support zillion _end_ libraries, sure, but they do need to have at least a very good set of standard libraries, equivalent of Cublas, Curand etc. And they don't. My work recently had me working with rocRAND (Rocm's answer to Curand). It was frankly pretty bad- the design, performance (50% slower in places that don't make any sense…

> generating random numbers You can't bench implementations of random numbers against each other purely on execution speed. A better algorithm (better statistical properties) will be slower.

Yeah. In this instance, I was talking about the same algorithm (phillox), the difference is purely in implementation.

Re: Intel CEO: 'The entire industry is motivated to eliminate the CUDA market'

#350

Post by AnthonyMouse »

latchkey wrote: post #191
AnthonyMouse wrote: post #129

Earlier quoted context omitted.

8GB is enough for Stable Diffusion or Llama 13B q4. They're outdated, but non-outdated GPUs are still expensive , so they're all many people can afford. > I'll put you in touch with someone to buy 130,000 of these cards, and you can sell them to every college kid out there... Just sell them on eBay? They still go for $50-$100 each, so you're sitting on several million dollars worth of GPUs. > I'd personally rather se…

> Growing the community is how you catch up. Agreed 100%. > That doesn't happen if people can't afford the only GPUs you support. On this part, we are going to have to agree to disagree. I feel like being able to at least affordably rent time on the high end GPUs is another alternative to buying them. As I mentioned above, that is something I'm actively working on.

> I feel like being able to at least affordably rent time on the high end GPUs is another alternative to buying them.

There are two problems with this.

The first is high demand. GPU time on a lot of cloud providers is sold out.

The second is that this costs money at all, vs. using the GPU you already have. "Need for credit card" is a barrier to hobbyists and you want hobbyists, because they become contributors or get introduced to the technology and then go on to buy one of your more expensive GPUs.

You want the barrier to adoption to be level with the ground.

Post reply on HN