Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hacked Nvidia 4090 GPU driver to enable P2P

github.com

Post reply on HN
Print view View original

151–160 of 365 posts

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#151

Post by the8472 »

stale2002 wrote: post #140
the8472 wrote: post #137

Earlier quoted context omitted.

Voldemort is fictional and so are bumbling wizard apprentices. Toy-level, not-yet-harmful AIs on the other hand are real. And so are efforts to make them more powerful. So the proposition that more powerful AIs will exist in the future is far more likely than an evil super wizard coming into existence. And I don't think literal 5-word-magic-incantation mind control is essential for an AI to be dangerous. More subtle…

> Toy-level, not-yet-harmful AIs on the other hand are real. A computer that can cause harm is much different than the absurd claims that I am disagreeing with. The extraordinary claims that are equivalent to saying that the imperious curse exists would be the magic computers that create diamond nanobots and mind control humans. > that more powerful AIs will exist in the future Bad argument. Non safe Boxes exist in r…

Instead of gesturing at flawed analogies, let's return to the actual issue at hand. Do you think that agents more intelligent than humans are impossible or at least extremely unlikely to come into existence in the future? Or that such super-human intelligent agents are unlikely to have goals that are dangerous to humans? Or that they would be incapable of pursuing such goals?

Also, it seems obvious that the standard of evidence that "AI could cause extinction" can't be observing an extinction level event, because at that point it would be too late. Considering that preventive measures would take time and safety margin, which level of evidence would be sufficient to motivate serious countermeasures?

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#152

Post by cjbprime »

stale2002 wrote: post #123
cjbprime wrote: post #88

Earlier quoted context omitted.

It doesn't sound like you gave serious thought to the arguments. The AGI doesn't need to hack robots. It has superhuman persuasion, by definition; it can "hack" (enough of) the humans to achieve its goals.

AI mind control abilities are also on the level of an extraordinary claim, that requires extraordinary evidence. It's on the level of "we better regulate wooden sticks so Voldemort doesn't use the imperious curse on us!". That's how I treat such claims. I treat them the same as someone literally talking about magic from Harry potter. There isn't nothing that would make me believe that. But it requires actual evidence…

What do you think mind control is? Think President Trump but without the self-defeating flaws, with an ability to stick to plans, and most importantly the ability to pay personal attention to each follower to further increase the level of trust and commitment. Not Harry Potter.

People will do what the AI says because it is able to create personal trust relationships with them and they want to help it. (They may not even realize that they are helping an AI rather than a human who cares about them.)

The normal ways that trust is created, not magical ones.

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#153

Post by liuliu »

andersa wrote: post #80

Incredible! I'd been wondering if this was possible. Now the only thing standing in the way of my 4x4090 rig for local LLMs is finding time to build it. With tensor parallelism, this will be both massively cheaper and faster for inference than a H100 SXM. I still don't understand why they went with 6 GPUs for the tinybox. Many things will only function well with 4 or 8 GPUs. It seems like the worst of both worlds now…

6 seems reasonable. 128 Lanes from ThreadRipper needs to have a few for network and NVMe (4x NVMe would be x16 lanes, and 10G network would be another x4 lanes).

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#155

Post by wmf »

CamperBob2 wrote: post #122
jstanley wrote: post #30

What does P2P mean in this context? I Googled it and it sounds like it means "peer to peer", but what does that mean in the context of a graphics card?

The correct term, and the one most people would have used in the past, is "bus mastering."

PCIe isn't a bus and it doesn't really have a concept of mastering. All PCI DMA was based on bus mastering but P2P DMA is trickier than normal DMA.

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#156

Post by renewiltord »

andersa wrote: post #129
renewiltord wrote: post #124

Earlier quoted context omitted.

I’ve got a few 4090s that I’m planning on doing this with. Would appreciate even the smallest directional tip you can provide on splitting the model that you believe is likely to work.

The split is done automatically by the inference engine if you enable tensor parallelism. TensorRT-LLM, vLLM and aphrodite-engine can all do this out of the box. The main thing is just that you need either 4 or 8 GPUs for it to work on current models.

Thank you! Can I run with 2 GPUs or with heterogeneous GPUs that have same RAM? I will try. Just curious if you already have tried.

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#157

Post by wmf »

namibj wrote: post #57

And here I thought (PCIe) P2P was there since SLI dropped the bridge (for the unfamiliar, it looks and acts pretty much like an NVLink bridge for regular PCIe slot cards that have NVLink, and was used back in the day to share framebuffer and similar in high-end gaming setups).

SLI was dropped years ago so there's no need for gaming cards to communicate at all.

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#158

Post by mhh__ »

userbinator wrote: post #76

I wish more hardware companies would publish more documentation and let the community figure out the rest, sort of like what happened to the original IBM VGA (look up "Mode X" and the other non-BIOS modes the hardware is actually capable of - even 800x600x16!) Sadly it seems the majority of them would rather tightly control every aspect of their products' usage since they can then milk the userbase for more $$$, but…

nvidia's software is their moat

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#159

Post by andersa »

renewiltord wrote: post #156
andersa wrote: post #129

Earlier quoted context omitted.

The split is done automatically by the inference engine if you enable tensor parallelism. TensorRT-LLM, vLLM and aphrodite-engine can all do this out of the box. The main thing is just that you need either 4 or 8 GPUs for it to work on current models.

Thank you! Can I run with 2 GPUs or with heterogeneous GPUs that have same RAM? I will try. Just curious if you already have tried.

2 GPUs works fine too, as long as your model fits. Using different GPUs with same VRAM however, is highly highly sketchy. Sometimes it works, sometimes it doesn't. In any case, it would be limited by the performance of the slower GPU.

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#160

Post by varelse »

Aerroon wrote: post #42
logicchains wrote: post #11

Earlier quoted context omitted.

Looks like we're only a few years away from a bona fide cyberpunk dystopia, in which only governments and megacorps are allowed to use AI, and hackers working on their own hardware face regular raids from the authorities.

I find it baffling that ideas like "govern compute" are even taken seriously. What the hell has happened to the ideals of freedom?! Does the government own us or something?

[dead]
Post reply on HN