Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hacked Nvidia 4090 GPU driver to enable P2P

github.com

Post reply on HN
Print view View original

41–50 of 365 posts

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#41

Post by s2l »

the8472 wrote: post #28
tomoyoirl wrote: post #22

Earlier quoted context omitted.

> In the sense that any other government regulation is also ultimately backed by the state's monopoly on legal use of force when other measures have failed. Good point. He was only (“only”) really calling for international cooperation and literal air strikes against big datacenters that weren’t cooperating. This would presumably be more of a no-knock raid, breaching your door with a battering ram and throwing tear ga…

... after regulation, court orders and fines have failed. Which under the premise that AGI is an existential threat would be far more reasonable than many other reasons for raids. If the premise is wrong we won't need it. If society coordinates to not do the dangerous thing we won't need it. The argument is that only in the case where we find ourselves in the situation where other measures have failed such uses of fo…

Time to publish the next book in "Stealing the network" series.

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#42

Post by Aerroon »

logicchains wrote: post #11
clbrmbr wrote: post #10

If we end up with a compute governance model of AI control [1], this sort of thing could get your door kicked in by the CEA (Compute Enforcement Agency). [1] https://podcasts.apple.com/us/podcast/ai-safety-fundamentals...

Looks like we're only a few years away from a bona fide cyberpunk dystopia, in which only governments and megacorps are allowed to use AI, and hackers working on their own hardware face regular raids from the authorities.

I find it baffling that ideas like "govern compute" are even taken seriously. What the hell has happened to the ideals of freedom?! Does the government own us or something?

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#43

Post by squarra »

jagrsw wrote: post #3

Was it George himself, or a person working for a bounty that was set up by tinycorp? Also, a question for those knowledgeable about the PCI subsys: it looked like something NVIDIA didn't care about, rather than something they actively wanted to prevent, no?

He also documented his progress on the tinygrad discord

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#44

Post by raxxorraxor »

the8472 wrote: post #28
tomoyoirl wrote: post #22

Earlier quoted context omitted.

> In the sense that any other government regulation is also ultimately backed by the state's monopoly on legal use of force when other measures have failed. Good point. He was only (“only”) really calling for international cooperation and literal air strikes against big datacenters that weren’t cooperating. This would presumably be more of a no-knock raid, breaching your door with a battering ram and throwing tear ga…

... after regulation, court orders and fines have failed. Which under the premise that AGI is an existential threat would be far more reasonable than many other reasons for raids. If the premise is wrong we won't need it. If society coordinates to not do the dangerous thing we won't need it. The argument is that only in the case where we find ourselves in the situation where other measures have failed such uses of fo…

To be honest, I see summoning the threat of AGI to pose an existential threat to be on the level with lizard people on the moon. Great for sci-fi, bad distraction for policy making and addressing real problems.

The real war, if there is one, is about owning data and collecting data. And surprisingly many people fall for distractions while their LLM fails at basic math. Because it is a language model of course...

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#45

Post by Aerroon »

the8472 wrote: post #28
tomoyoirl wrote: post #22

Earlier quoted context omitted.

> In the sense that any other government regulation is also ultimately backed by the state's monopoly on legal use of force when other measures have failed. Good point. He was only (“only”) really calling for international cooperation and literal air strikes against big datacenters that weren’t cooperating. This would presumably be more of a no-knock raid, breaching your door with a battering ram and throwing tear ga…

... after regulation, court orders and fines have failed. Which under the premise that AGI is an existential threat would be far more reasonable than many other reasons for raids. If the premise is wrong we won't need it. If society coordinates to not do the dangerous thing we won't need it. The argument is that only in the case where we find ourselves in the situation where other measures have failed such uses of fo…

>If the premise is wrong we won't need it. If society coordinates to not do the dangerous thing we won't need it.

But the idea that this use of force is okay itself increases danger. It creates the situation that actors in the field might realize that at some point they're in danger of this and decide to do a first strike to protect themselves.

I think this is why anti-nuclear policy is not "we will airstrike you if you build nukes" but rather "we will infiltrate your network and try to stop you like that".

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#46

Post by zamadatix »

ot1138 wrote: post #35
__alexs wrote: post #34

Earlier quoted context omitted.

It means you can send data from the memory of 1 GPU to another GPU without going via RAM. https://xilinx.github.io/XRT/master/html/p2p.html

Is this really efficient or practical? My understanding is that the latency required to copy memory from CPU or RAM to GPU negates any performance benefits (much less running over a network!)

Peer to peer as in one pcie slot directly to another without going through the CPU/RAM, not peer to peer as in one PC to another over the network port.

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#47

Post by llm_trw »

ot1138 wrote: post #35
__alexs wrote: post #34

Earlier quoted context omitted.

It means you can send data from the memory of 1 GPU to another GPU without going via RAM. https://xilinx.github.io/XRT/master/html/p2p.html

Is this really efficient or practical? My understanding is that the latency required to copy memory from CPU or RAM to GPU negates any performance benefits (much less running over a network!)

Yes, the point here is that you do a direct write from one cards memory to the other using PCIe.

In older NVidia cards this could be done through a faster link called NVLink but the hardware for that was ripped out of consumer grade cards and is only in data center grade cards now.

Until this post it seemed like they had ripped all such functionality of their consumer cards, but it looks like you can still get it working at lower speeds using the PCIe bus.

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#48

Post by im3w1l »

the8472 wrote: post #28
tomoyoirl wrote: post #22

Earlier quoted context omitted.

> In the sense that any other government regulation is also ultimately backed by the state's monopoly on legal use of force when other measures have failed. Good point. He was only (“only”) really calling for international cooperation and literal air strikes against big datacenters that weren’t cooperating. This would presumably be more of a no-knock raid, breaching your door with a battering ram and throwing tear ga…

... after regulation, court orders and fines have failed. Which under the premise that AGI is an existential threat would be far more reasonable than many other reasons for raids. If the premise is wrong we won't need it. If society coordinates to not do the dangerous thing we won't need it. The argument is that only in the case where we find ourselves in the situation where other measures have failed such uses of fo…

> I'm not seeing the odiousness of the proposal. If bio research gets commodified and easy enough that every kid can build a new airborne virus in their basement we'd need raids on that too.

Either you create even better bio research to neutralize said viruses... or you die trying...

Like if you go with the raid strategy and fail to raid just one terrorist that's it, game over.

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#49

Post by the8472 »

raxxorraxor wrote: post #44
the8472 wrote: post #28

Earlier quoted context omitted.

... after regulation, court orders and fines have failed. Which under the premise that AGI is an existential threat would be far more reasonable than many other reasons for raids. If the premise is wrong we won't need it. If society coordinates to not do the dangerous thing we won't need it. The argument is that only in the case where we find ourselves in the situation where other measures have failed such uses of fo…

To be honest, I see summoning the threat of AGI to pose an existential threat to be on the level with lizard people on the moon. Great for sci-fi, bad distraction for policy making and addressing real problems. The real war, if there is one, is about owning data and collecting data. And surprisingly many people fall for distractions while their LLM fails at basic math. Because it is a language model of course...

Freely flying through the sky on wings was scifi before the wright brothers. Something sounding like scifi is not a sound argument that it won't happen. And unlike lizard people we do have exponential curves to point at. Something stronger than a vibes-based argument would be good.

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#50

Post by throw101010 »

mtlynch wrote: post #7
jagrsw wrote: post #3

Was it George himself, or a person working for a bounty that was set up by tinycorp? Also, a question for those knowledgeable about the PCI subsys: it looked like something NVIDIA didn't care about, rather than something they actively wanted to prevent, no?

Commits are by geohot, so it looks like George himself.

I've seen him work on tinygrad on his Twitch livestream couple times, so more than likely him indeed.
Post reply on HN