271–280 of 365 posts
WTF is P2P?
https://developer.nvidia.com/gpudirect#:~:text=LEARN%20MORE%....
> GPUDirect Peer to Peer > Enables GPU-to-GPU copies as well as loads and stores directly over the memory fabric (PCIe, NVLink). GPUDirect Peer to Peer is supported natively by the CUDA Driver. Developers should use the latest CUDA Toolkit and drivers on a system with two or more compatible devices.
Earlier quoted context omitted.
That's a poor analogy. The feature is built in to the cards that consumers bought, but Nvidia is disabling it via software. That's why a hacked driver can enable it again. The resident in your analogy is just freeloading off the contractor's transformer. Nvidia does this so that customers that need that feature are forced to buy more expensive systems instead of building a solution with the cheaper "consumer-grade" c…
This isn’t even the first time a hacked driver has been used to unlock some HW feature - https://github.com/DualCoder/vgpu_unlock
I wish more hardware companies would publish more documentation and let the community figure out the rest, sort of like what happened to the original IBM VGA (look up "Mode X" and the other non-BIOS modes the hardware is actually capable of - even 800x600x16!) Sadly it seems the majority of them would rather tightly control every aspect of their products' usage since they can then milk the userbase for more $$$, but…
nvidia's software is their moat
Earlier quoted context omitted.
Oooo, got a link to one of these fabric boards? I've been playing with stupid PCIe tricks but that's a new one on me.
https://www.amazon.com/gp/product/B07DMNJ6QM/ i used to use this one when i had all (three of my) nvme -> 4x sata boardlets and therefore could not fit a GPU in a PCIe slot due to the cabling mess.
I thought we were using "fabric" to mean "switching matrix".
I wish more hardware companies would publish more documentation and let the community figure out the rest, sort of like what happened to the original IBM VGA (look up "Mode X" and the other non-BIOS modes the hardware is actually capable of - even 800x600x16!) Sadly it seems the majority of them would rather tightly control every aspect of their products' usage since they can then milk the userbase for more $$$, but…
Then they couldn't charge different customers different amounts for the same HW. It's not a win for everyone.
Is this one of those features that's disabled on consumer cards for market segmentation?
I am sure many will disagree-vote me, but I want to see this practice in consumer devices either banned or very heavily taxed.
https://github.com/NVIDIA/open-gpu-kernel-modules/commit/1f4...
Earlier quoted context omitted.
> It doesn't beat RTX 4090 when it comes to actual LLM inference speed Sure, whisper.cpp is not an LLM. The 4090 can't even do inference at all on anything over 24GB, while ASi can chug through it even if slightly slower. I wonder if with https://github.com/tinygrad/open-gpu-kernel-modules (the 4090 P2P patches) it might become a lot faster to split a too-large model across multiple 4090s and still outperform ASi (at…
> The 4090 can't even do inference at all on anything over 24GB, while ASi can chug through it even if slightly slower. Common LLM runners can split model layers between VRAM and system RAM; a PC rig with a 4090 can do inference on models larger than 24G. Where the crossover point where having the whole thing on Apple Silicon unified memory vs. doing split layers on a PC with a 4090 and system RAM is, I don't know, b…
Sure and ASi can do inference on models larger than the Unified Memory if you account for streaming the weights from the SSD on-demand. That doesn't mean it's going to be as fast as keeping the whole thing in RAM, although ASi SSDs are probably not particularly bad as far as SSDs go.
Earlier quoted context omitted.
I find it baffling that ideas like "govern compute" are even taken seriously. What the hell has happened to the ideals of freedom?! Does the government own us or something?
If AI is actually capable of fulfilling all the capabilities suggested by people who believe in the singularity, it has far more capacity for harm than nuclear weapons. I think most people who are strongly pro-AI/pro-acceleration - or, at any rate, not anti-AI - believe that either (A) there is no control problem (B) it will be solved (C) AI won't become independent and agentic (i.e. it won't face evolutionary pressu…