271–280 of 380 posts
Earlier quoted context omitted.
It was called Larrabee and XeonPhi, they botched it, and the only thing left from that effort is AVX.
I used to play with these toys 7-8 years ago. We tried everything, and it was bad at it all. Traditional compute? The cores were too weak. Number crunching? Okay-ish but gpus were better. Useless stuff.
Earlier quoted context omitted.
I am unfamiliar with the implementation, but why would it be difficult to implement a Cuda-compatible software layer on top of other platforms? This would be the first step. Then, if we want to move away from Cuda into hardware that's as ubiquitous and performant as Nvidia's (or better), someone would need to write an abstraction layer that's more convenient to use than Cuda. I did play a little bit with Cuda and Ope…
Worth noting that cuda seems prone to using inline ptx assembly and that latter is really obnoxious to deal with on other platforms. Implementing programming languages and runtimes is pretty difficult in general. Note that cuda doesn't have the same semantics as c++ despite looking kind of similar. Wherever you differ from expected behaviour people consider it a bug, and implementing based on cuda's docs wouldn't get…
Earlier quoted context omitted.
but it's all inference, and most of Nvidias moat is in training afaik.
People have really bizarre overdramatic misunderstandings of llama.cpp because they used it a few times to cook their laptop. This one really got me giggling though.
This is a leading-edge software library that provides a huge boost for non-Nvidia hardware in terms of inference capability with quantized models.
If you don't understand that, then you have missed an important development in the space of machine learning.
Earlier quoted context omitted.
> But FWIW the incumbent adopts it and dominates anyway. (Though you now are technically "untethered.") That's assuming the incumbent's advantage isn't rooted in the lock-in. If ML was suddenly untethered from CUDA, now you're competing on hardware. Intel would still have mediocre GPUs, but AMD's are competitive, and Intel's could be in the near future if they execute competently. The open standard doesn't automatica…
If the software side no longer mattered, I would expect all three vendors would magically start competing on available RAM. A slower card with double today's RAM would absolutely sell.
Absolutely, SQL analytics people (like me) have been itching for a viable GPU for analytics for years now. The price/performance just isn't there yet because there's such a bias towards high compute and low memory.
Earlier quoted context omitted.
> It's rare to find an operating system that doesn't implement the POSIX API. Except that it has barelly improved beyond CLI and daemons, still thinks terminals are the only hardware, everything else that matters isn't part of it, not even more modern networking protocols that aren't exposed in socket configurations.
Its purpose was to create compatibility between Unix vendors so developers could write software compatible with different flavors. The primary market for Unix vendors is servers, which to this day are still about CLI and daemons, and POSIX systems continue to have dominant market share in that market.
Alongside a cloud shell, which yeah, we now have a VT100 running on a browser window.
There is a reason why there are USENIX papers on the loss of POSIX relevance.
As another commenter said, it's CUDA. Intel and AMD and whoever can turn out chips reasonably fast, but nobody gets that it's the software and ecosystem. You have to out-compete the ecosystem. You can pick up a used Mi100 that performs almost like an A100 for 5x less money on eBay for example. Why is it 5x less? Because the software incompatibilities mean you'll spend a ton of time getting it to work compared to an N…
> but nobody gets that it's the software and ecosystem. Really? Because I’m confident Intel knows exactly what it’s about. Have you looked at their contributions to Linux and open source in general? They employ thousands of software developers.
Oh, wait…
Earlier quoted context omitted.
A subset that lacks pointers, which makes compute shaders a toy language next to CUDA.
Vulkan 1.3 has pointers, thanks to buffer device address[1]. It took a while to get there, and earlier pointer support was flawed. I also don't know of any major applications that use this. Modern Vulkan is looking pretty good now. Cooperative matrix multiplication has also landed (as a widely supported extension), and I think it's fair to say it's gone past OpenCL. Whether we get significant adoption of all this I t…
> [1] https://community.arm.com/arm-community-blogs/b/graphics-gam...
"Using a pointer in a shader - In Vulkan GLSL, there is the GL_EXT_buffer_reference extension "
That extension is utter garbage. I tried it. It was the last thing I tried before giving up on GLSL/Vulkan and switching to CUDA. It was the nail in the coffin that made me go "okay, if that's the best Vulkan can do, then I need to switch to CUDA". It's incredibly cumbersome, confusing and verbose.
What's needed are regular, simple, C-like pointers.
Earlier quoted context omitted.
>finds a CEO who is as technical and strategic, as opposed to the bean-counters Did you just call Gelsinger a "non-technical"? wow, how out of touch with reality >Gelsinger first joined Intel at 18 years old in 1979 just after earning an associate degree from Lincoln Tech.[9] He spent much of his career with the company in Oregon,[12] where he maintains a home.[13] In 1987, he co-authored his first book about program…
If you read carefully you’ll see the comment is “as technical and strategic.” That’s very different from “non-technical.” He was clearly capable of leading a team to develop a new processor but that’s not the issue here.
He has wanted GPGPU or at least massively parallel compute for a long time and says it would be 10 years of work to make that happen.
And secondly, do you really think he's wrong about this part: "As inferencing occurs, hey, once you've trained the model… There is no CUDA dependency," Gelsinger continued. "It's all about, can you run that model well?"
Why run inferencing on an expensive H100? Gaudi seems like a pretty good idea.
Intel should have an advantage here as MKL and OneAPI are quite good, but the AMD’s roc libraries are not very good.