Any updates here to make this work on more modern llms?
Petals: Run LLMs at home, BitTorrent-style
21–30 of 42 posts
Re: Petals: Run LLMs at home, BitTorrent-style
#22This is awesome. Now we also need distributed training of models!
Re: Petals: Run LLMs at home, BitTorrent-style
#23Imagine if there was some kind of way to cryptographically 'prove' your GPU is doing some kind of 'work' here and contributing to the network. You could even hand out some kind of digital 'currency' to the people proportional to the amount of work their doing!
As we've seen, that's a solution looking for a problem.
More like a problem looking for more problem.
Re: Petals: Run LLMs at home, BitTorrent-style
#24Earlier quoted context omitted.
Can't wait to run a Sybil farm of subsidized proxy nodes and log all the juicy prompts.
I do not know what/who Sybil is but yes, there is a lot of plumbing in order to make sure that host nodes are ZDR* compliant and much more. Zero visibility of source prompt. Also, this is the reason I want to start with trust based groups only - invite people you already know. I have friends who have Macs with 32GB or 48GB unified memory but sending prompts to them is not a out of the box thing. * typo
> A Sybil attack is a type of attack on a computer network service in which an attacker subverts the service's reputation system by creating a large number of pseudonymous identities and uses them to gain a disproportionately large influence.
Re: Petals: Run LLMs at home, BitTorrent-style
#25This is super cool technology, but will be abused to death
Re: Petals: Run LLMs at home, BitTorrent-style
#26Related. Others?
Run LLMs at home, BitTorrent‑style - https://news.ycombinator.com/item?id=37546810 - Sept 2023 (125 comments)
Re: Petals: Run LLMs at home, BitTorrent-style
#27Imagine if there was some kind of way to cryptographically 'prove' your GPU is doing some kind of 'work' here and contributing to the network. You could even hand out some kind of digital 'currency' to the people proportional to the amount of work their doing!
I've occasionally written comments about the difficult of achieving proof-of-useful-work mechanisms and I should probably find or write a standard post about this question.
In order to be as decentralized as Bitcoin and related PoW mechanisms, a proof-of-work mechanism should have
* the ability to create an unbounded number of instances of the problem deterministically from numeric seeds
* the ability to scale difficulty up and down using a difficulty parameter
* in a way where anyone can easily confirm that a given problem instance corresponds to a given seed, and anyone can easily confirm that a given solution correctly solves a given problem instance
I'm not aware of a proof-of-useful-work mechanism that meets these criteria. (I'm also not aware of an argument that it's impossible to have one. Zero-knowledge proofs might actually go far toward making it possible in the future. but it's kind of complicated.)
Re: Petals: Run LLMs at home, BitTorrent-style
#28This is awesome. Now we also need distributed training of models!
Distributed training is much harder than distributed inference but not impossible. See the recent development of DiLoCo at Nous Research and Prime Intellect.
Re: Petals: Run LLMs at home, BitTorrent-style
#29I did a triple take. Many years old but an early ground breaking project. Has some exploratoy ideas a year or two ago last I looked. Crazy to see Petals at 16 frontage (at time of writing)
Re: Petals: Run LLMs at home, BitTorrent-style
#30It's an interesting concept but the timing is probably too early. If more people had reliable low latency gigabit or ideally 10gbit throughput, then it might start to approach feasibility. There's other blockers too, but that springs to mind immediately. It's cool to imagine a planet wide neural network interconnected with fiber - the nervous system of a planetary intelligence. But perhaps mushrooms do that already (…
Not a neural network though ;) https://biologyinsights.com/do-fungi-have-a-nervous-system-t...