141–150 of 162 posts
Earlier quoted context omitted.
Specifically the clause is that you cannot use their consumer cards (e.g. RTX 4090) in datacenters.
That's why we run all of our ML workloads in a distributed GPU cluster located in every employee's house
Does it make sense to buy Nvidia GPUs as a linux user in 2024 anyway? I thought Nvidia has abysmal linux support, if you don't have Nvidia GPU what's the point of LibreCUDA?
Earlier quoted context omitted.
> But in the long run open source wins because once an open source alternative exists it is just the better option. This is just wishful thinking. Anything close to real professional use, not related to IT, and closed source is king: office work, CAD, video editing, music production, and those domains immediately came to mind. Nowhere there open source can seriously challenge commercial, closed sourced competitors. Y…
fucking unreal engine 5 is open source, dawg!
I think the point of open cuda is to run it on non NVIDIA gpus. Once you have to buy NVIDIA gpus what’s the point. If we had true you competition I think it would be far easier to buy devices with more vram and thus we might be able to run llama 405b someday locally. Once you already bought the NVIDIA cards what’s the point
Some of us are running llama 405B locally already. All my GPUs are ancient Nvidai GPUs. IMO, the point of an open cuda is to force Nvidia to stop squeezing us. You get more performance for the buck for AMD. If I could run cuda on AMD, I would have bought new AMD gpus instead. Have enough people do that and Nvidia might take note and stop squeezing us for cash.
Cool to see one of these in C, particularly if it can be binary compatible. Why not s/libreCuInit/cuInit? If you are interested in open source runtimes, tinygrad has them in Python for both AMD and NVIDIA, speaking directly to the kernel through ioctls and poking the command queues. https://github.com/tinygrad/tinygrad/blob/master/tinygrad/ru... https://github.com/tinygrad/tinygrad/blob/master/tinygrad/ru...
That's interesting. This looks like you've bypassed the rocm userspace stack entirely. I've been looking for justification to burn libhsa.so out of the dependency graph for running llvm compiled kernels on amdgpu for ages now. I didn't expect roct to be similarly easy to drop but that's a clear sketch of how to build a statically linked freestanding x64 / gcn blob. Excellent. (I want a reference implementation of run…
The HSA parsing MEC firmware running on the GPUs is riddled with bugs, fortunately you can bypass 90% of it using PM4, which is pretty much direct sets of the GPU registers. That's what tinygrad does.
AMD's software is a really sad state. They don't have consumer GPUs in CI, they have no fuzz testing, and instead of root causing bugs they seem to just twiddle things until the application works.
Between our PM4 backend and disabling CWSR, our AMD GPUs are now pretty stable.
Earlier quoted context omitted.
That's interesting. This looks like you've bypassed the rocm userspace stack entirely. I've been looking for justification to burn libhsa.so out of the dependency graph for running llvm compiled kernels on amdgpu for ages now. I didn't expect roct to be similarly easy to drop but that's a clear sketch of how to build a statically linked freestanding x64 / gcn blob. Excellent. (I want a reference implementation of run…
We didn't just bypass all of ROCm, we bypassed HSA! The HSA parsing MEC firmware running on the GPUs is riddled with bugs, fortunately you can bypass 90% of it using PM4, which is pretty much direct sets of the GPU registers. That's what tinygrad does. AMD's software is a really sad state. They don't have consumer GPUs in CI, they have no fuzz testing, and instead of root causing bugs they seem to just twiddle things…
There is a lot one cannot wisely say about the political structures which give rise to the behaviour of the rocm toolchain. Some of the implementation choices reified in the code are far divorced from what I consider reasonable, but one does not take a hatchet to other people's work.
This makes fixing rocm primarily a people oriented problem to which compiler engineers are not necessarily temperamentally suited. I note that a technically inept but politically minded engineer can do extremely widespread damage in a friendly non-toxic collaborative environment.
Cutting the egregiously broken parts of rocm from one's dependency graph is a solid workaround.