Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Petals: Run LLMs at home, BitTorrent-style

petals.dev

Post reply on HN
Print view View original

11–20 of 42 posts

Re: Petals: Run LLMs at home, BitTorrent-style

#11

Post by hahahaa »

N_Lens wrote: post #4

It's an interesting concept but the timing is probably too early. If more people had reliable low latency gigabit or ideally 10gbit throughput, then it might start to approach feasibility. There's other blockers too, but that springs to mind immediately. It's cool to imagine a planet wide neural network interconnected with fiber - the nervous system of a planetary intelligence. But perhaps mushrooms do that already (…

Too early too in that later (hopefully) each node can serve a full model so it is then a simpler share tit for tat of complete models.

Re: Petals: Run LLMs at home, BitTorrent-style

#12

Post by mcpindex-ai »

brainless wrote: post #8
inigyou wrote: post #6

Earlier quoted context omitted.

Can't wait to run a Sybil farm of subsidized proxy nodes and log all the juicy prompts.

I do not know what/who Sybil is but yes, there is a lot of plumbing in order to make sure that host nodes are ZDR* compliant and much more. Zero visibility of source prompt. Also, this is the reason I want to start with trust based groups only - invite people you already know. I have friends who have Macs with 32GB or 48GB unified memory but sending prompts to them is not a out of the box thing. * typo

[flagged]

Re: Petals: Run LLMs at home, BitTorrent-style

#15

Post by atum47 »

maccam912 wrote: post #14

The recently discussed https://meshllm.cloud/ is the one I've been playing with but don't have the hardware to try with a model split between nodes, which apparently is supported and just not part of the public demo.

Can't even scroll the website on mobile, too much going on

Re: Petals: Run LLMs at home, BitTorrent-style

#16

Post by bravoetch »

crowcroft wrote: post #13

Imagine if there was some kind of way to cryptographically 'prove' your GPU is doing some kind of 'work' here and contributing to the network. You could even hand out some kind of digital 'currency' to the people proportional to the amount of work their doing!

As we've seen, that's a solution looking for a problem.

Re: Petals: Run LLMs at home, BitTorrent-style

#18

Post by gatuamgb »

lostmsu wrote: post #2

Not gonna work, will be starved by inter node bandwidth.

You can easily build a localized, access-controlled compute grid using Apple Silicon hardware. Haven't tested but as far as I can see, the key to making it performant would be keeping full models resident on individual nodes rather than attempting to distribute the tensor processing across the network.

Re: Petals: Run LLMs at home, BitTorrent-style

#19

Post by behnamoh »

N_Lens wrote: post #4

It's an interesting concept but the timing is probably too early. If more people had reliable low latency gigabit or ideally 10gbit throughput, then it might start to approach feasibility. There's other blockers too, but that springs to mind immediately. It's cool to imagine a planet wide neural network interconnected with fiber - the nervous system of a planetary intelligence. But perhaps mushrooms do that already (…

Too early? the project has been going for years now...

Re: Petals: Run LLMs at home, BitTorrent-style

#20

Post by goda90 »

bravoetch wrote: post #16
crowcroft wrote: post #13

Imagine if there was some kind of way to cryptographically 'prove' your GPU is doing some kind of 'work' here and contributing to the network. You could even hand out some kind of digital 'currency' to the people proportional to the amount of work their doing!

As we've seen, that's a solution looking for a problem.

But what if the work is useful beyond just crunching numbers for the sake of proving it?
Post reply on HN