Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hacked Nvidia 4090 GPU driver to enable P2P

github.com

Post reply on HN
Print view View original

301–310 of 365 posts

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#301

Post by snakeyjake »

erikbye wrote: post #177
snakeyjake wrote: post #147

Earlier quoted context omitted.

You need to abandon your apocalyptic worldview keep up with the times my friend. Encryption export controls have been systematically dismantled to the point that they're practically non-existent, especially over the last three years. Pretty much the only encryption products you need permission to export are those specifically designed for integration into military communications networks, like Digital Subscriber Voic…

Are you on drugs or is your reading comprehension that poor? 1) I did not state a world view; I simply noted that restrictions for software do exist, and will for AI, as well. As the link from the other commenter show, they do in fact already exist. 2) Look up the definition of "apocalyptic", software restrictions are not within its bounds. 3) How the restrictions are enforced were not a subject in my comment. 4) We'…

I'm high on life, old chum!

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#302

Post by arghwhat »

segfaultbuserr wrote: post #69
arghwhat wrote: post #21

Earlier quoted context omitted.

It does seem like an oversight, but there's nothing "suboptimal non-default options" about iteven if the implementation posted here seems somewhat hastily hacked together.

> but there's nothing "suboptimal non-default options" about it If "bypassing the official driver to invoke the underlying hardware feature directly through source code modification (and incompatibilities must be carefully worked around by turning off IOMMU and large BAR, since the feature was never officially supported)" does not count as "suboptimal non-default options", then I don't know what counts as "suboptimal…

> bypassing the official driverto

The driver is not bypasses. This is a patch to the official open-source kernel-driver where the feature is added, which is how all upstream Linux driver development is done.

> to invoke the underlying hardware feature directly

Accessing hardware features directly is pretty much the sole job of a driver, and the only thing "bypassed" is some abstractions internal to the driver. Just means the patch would fail review in basis of codestyle, and on the basis of possibly only supporting one device family.

> through source code modification

That is a weird way to describe software engineering. Making the code available for further development is kind of the whole point of open source.

> turning off IOMMU

This is not a P2PDMA problem, and just a result of them not also adding the necessary IOMMU boilerplate, which would be added if the patch was done properly to be upstreamed.

> large BAR

This is an expected and "optimal" system requirement.

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#303

Post by numpad0 »

wtallis wrote: post #238
numpad0 wrote: post #230

Earlier quoted context omitted.

PCIe P2P still has to go up to a central hub thing and back because PCIe is not a bus. That central hub thing is made by very few players(most famously PLX Technologies) and it costs a lot.

PCIe p2p transactions that end up routed through the CPU's PCIe root complex still have performance advantages over split transactions using the CPU's DRAM as an intermediate buffer. Separate PCIe switches are not necessary except when the CPU doesn't support routing p2p transactions, which IIRC was not a problem on anything more mainstream than IBM POWER.

Maybe not strictly necessary, but a separate PCIe backplane just for P2P bandwidth bypasses topology and bottleneck mess[1][2] of PC platform altogether and might be useful. I suspect this was the original premise for NVLink too.

1: https://assets.hardwarezone.com/img/2023/09/pre-meteror-lake...

2: https://www.gigabyte.com/FileUpload/Global/MicroSite/579/inn...

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#305

Post by aceazzameen »

metadat wrote: post #131
magicalhippo wrote: post #106

Earlier quoted context omitted.

These are part of the chip, thus microscopic and very inaccessible. There are some good images here[1] of various such fuses, both pristine and blown. Here's[2] a more detailed writeup examining one type. It's not something you fix with a soldering iron. [1]: https://semiengineering.com/the-benefits-of-antifuse-otp/ [2]: https://www.eetimes.com/a-look-at-metal-efuses/

I miss the days when you could do things like connecting the L5 bridges on the surface of the AMD Athlon XP Palomino [0] CPU packaging with a silver trace pen to transform them into fancier SMP multi-socket capable Athlon MPs, e.g. Barton [1]. https://arstechnica.com/civis/threads/how-did-you-unlock-you... Some folks even got this working with only a pencil, haha. Nowadays, silicon designers have found highly effecti…

I'm one of those folks that did it with a pencil. Haha. Maybe I was lucky? That was my first overclock and it ran pretty well.

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#306

Post by imtringued »

wtallis wrote: post #235
papichulo2023 wrote: post #216

Earlier quoted context omitted.

No many consumer mobo support 192GB DDR5.

If it supports DDR5 at all, then it should be at most a firmware update away from supporting 48GB dual-rank DIMMs. There are very few consumer motherboards that only have two DDR5 slots; almost all have the four slots necessary to accept 192GB. If you are under the impression that there's a widespread limitation on consumer hardware support for these modules, it may simply be due to the fact that 48GB modules did not…

You don't want to use more than two slots because you only have two memory channels. The overclocking potential of DDR5 is extremely high when you only run two DIMMs. All the way up to 8000. Meanwhile if you go for populating all four slots, you are limited significantly below 5000. Almost a 50% performance drop if you are willing to overclock your RAM.

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#307

Post by wtallis »

imtringued wrote: post #306
wtallis wrote: post #235

Earlier quoted context omitted.

If it supports DDR5 at all, then it should be at most a firmware update away from supporting 48GB dual-rank DIMMs. There are very few consumer motherboards that only have two DDR5 slots; almost all have the four slots necessary to accept 192GB. If you are under the impression that there's a widespread limitation on consumer hardware support for these modules, it may simply be due to the fact that 48GB modules did not…

You don't want to use more than two slots because you only have two memory channels. The overclocking potential of DDR5 is extremely high when you only run two DIMMs. All the way up to 8000. Meanwhile if you go for populating all four slots, you are limited significantly below 5000. Almost a 50% performance drop if you are willing to overclock your RAM.

If you want to run something that doesn't fit in 96GB of RAM, you'll get better performance from having enough RAM. Yes, having two dual-rank DIMMs per channel will force you to run at a slower speed, but it's still far faster than your SSD. The second slot per channel exists precisely because many people really do want to use it.

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#308

Post by [deleted] »

htrp wrote: post #38
andy99 wrote: post #26

Earlier quoted context omitted.

On one hand I'm strongly against letting that happen, on the other there's something romantic about the idea of smuggling the latest Chinese LLM on a flight from Neo-Tokyo to Newark in order to pay for my latest round of nervous system upgrades.

> On one hand I'm strongly against letting that happen, on the other there's something romantic about the idea of smuggling the latest Chinese LLM on a flight from Neo-Tokyo to Newark in order to pay for my latest round of nervous system upgrades. At least call it the 'Free City of Newark'

[deleted]

Re: Hacked Nvidia 4090 GPU driver to enable P2P

#310

Post by phyalow »

andy99 wrote: post #26
logicchains wrote: post #11

Earlier quoted context omitted.

Looks like we're only a few years away from a bona fide cyberpunk dystopia, in which only governments and megacorps are allowed to use AI, and hackers working on their own hardware face regular raids from the authorities.

On one hand I'm strongly against letting that happen, on the other there's something romantic about the idea of smuggling the latest Chinese LLM on a flight from Neo-Tokyo to Newark in order to pay for my latest round of nervous system upgrades.

This was always my favourite passage of Neuromancer: "THE JAPANESE HAD already forgotten more neurosurgery than the Chinese had ever known. The black clinics of Chiba were the cutting edge, whole bodies of technique supplanted monthly, and still they couldn’t repair the damage he’d suffered in that Memphis hotel. A year here and he still dreamed of cyberspace, hope fading nightly. All the speed he took, all the turns he’d taken and the corners he’d cut in Night City, and still he’d see the matrix in his sleep, bright lattices of logic unfolding across that colorless void. . . . The Sprawl was a long strange way home over the Pacific now, and he was no console man, no cyberspace cowboy. Just another hustler, trying to make it through. But the dreams came on in the Japanese night like livewire voodoo, and he’d cry for it, cry in his sleep, and wake alone in the dark, curled in his capsule in some coffin hotel, his hands clawed into the bedslab, temperfoam bunched between his fingers, trying to reach the console that wasn’t there.”
Post reply on HN