Run LLMs at home, BitTorrent‑style
petals.dev
Run LLMs at home, BitTorrent‑style
1–10 of 135 posts
Re: Run LLMs at home, BitTorrent‑style
#2The first question I had was "what are the economics?" From the FAQ:
Will Petals incentives be based on crypto, blockchain, etc.?
No, we are working on a centralized incentive system similar to the AI Horde kudos, even though Petals is a fully decentralized system in all other aspects. We do not plan to provide a service to exchange these points for money, so you should see these incentives as "game" points designed to be spent inside our system.
Petals is an ML-focused project designed for ML researchers and engineers, it does not have anything to do with finance. We decided to make the incentive system centralized because it is much easier to develop and maintain, so we can focus on developing features useful for ML researchers.
Re: Run LLMs at home, BitTorrent‑style
#3The first question I had was "what are the economics?" From the FAQ: Will Petals incentives be based on crypto, blockchain, etc.? No, we are working on a centralized incentive system similar to the AI Horde kudos, even though Petals is a fully decentralized system in all other aspects. We do not plan to provide a service to exchange these points for money, so you should see these incentives as "game" points designed…
> similar to the AI Horde kudos
What they are referencing, which is super cool and (IMO) criminally underused:
In fact, I can host a 13B-70B finetune in the afternoon if anyone on HN wants to test a particular one out:
Re: Run LLMs at home, BitTorrent‑style
#4The first question I had was "what are the economics?" From the FAQ: Will Petals incentives be based on crypto, blockchain, etc.? No, we are working on a centralized incentive system similar to the AI Horde kudos, even though Petals is a fully decentralized system in all other aspects. We do not plan to provide a service to exchange these points for money, so you should see these incentives as "game" points designed…
if that part could be replaced with any third party server it would be a tracker in BitTorrent analogy.
Re: Run LLMs at home, BitTorrent‑style
#5The first question I had was "what are the economics?" From the FAQ: Will Petals incentives be based on crypto, blockchain, etc.? No, we are working on a centralized incentive system similar to the AI Horde kudos, even though Petals is a fully decentralized system in all other aspects. We do not plan to provide a service to exchange these points for money, so you should see these incentives as "game" points designed…
[deleted]
Re: Run LLMs at home, BitTorrent‑style
#6The first question I had was "what are the economics?" From the FAQ: Will Petals incentives be based on crypto, blockchain, etc.? No, we are working on a centralized incentive system similar to the AI Horde kudos, even though Petals is a fully decentralized system in all other aspects. We do not plan to provide a service to exchange these points for money, so you should see these incentives as "game" points designed…
It's a shame that every decentralized projects needs to be compared to cryptocoins now
Re: Run LLMs at home, BitTorrent‑style
#7The first question I had was "what are the economics?" From the FAQ: Will Petals incentives be based on crypto, blockchain, etc.? No, we are working on a centralized incentive system similar to the AI Horde kudos, even though Petals is a fully decentralized system in all other aspects. We do not plan to provide a service to exchange these points for money, so you should see these incentives as "game" points designed…
The logical conclusion is that they (the models) will eventually be linked to crypto payments though. This is where Lightning becomes important...
Edit: To clarify, I'm not suggesting linking these Petal "tokens" to any payment system. I'm talking about, in general, calls to clusters of machine learning models, decentralized or not, will likely use crypto payments because it gives you auth and a means of payment.
I do think Petal is a good implementation of using decentralized compute for model use and will likely be valuable long term.
Re: Run LLMs at home, BitTorrent‑style
#8The first question I had was "what are the economics?" From the FAQ: Will Petals incentives be based on crypto, blockchain, etc.? No, we are working on a centralized incentive system similar to the AI Horde kudos, even though Petals is a fully decentralized system in all other aspects. We do not plan to provide a service to exchange these points for money, so you should see these incentives as "game" points designed…
> similar to the AI Horde kudos What they are referencing, which is super cool and (IMO) criminally underused: https://lite.koboldai.net/ https://tinybots.net/artbot https://aihorde.net/ In fact, I can host a 13B-70B finetune in the afternoon if anyone on HN wants to test a particular one out: https://huggingface.co/models?sort=modified&search=70B+gguf
> GGUF is a new format introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp. GGUF offers numerous advantages over GGML, such as better tokenisation, and support for special tokens. It is also supports metadata, and is designed to be extensible
is there a more canonical blogpost or link to learn more about the technical decisions here?
Re: Run LLMs at home, BitTorrent‑style
#9so given that GGML can serve like 100 tok/s on an M2 Max, and this thing advertises 6 tok/s distributed, is this basically for people with lower end devices?
Re: Run LLMs at home, BitTorrent‑style
#10The first question I had was "what are the economics?" From the FAQ: Will Petals incentives be based on crypto, blockchain, etc.? No, we are working on a centralized incentive system similar to the AI Horde kudos, even though Petals is a fully decentralized system in all other aspects. We do not plan to provide a service to exchange these points for money, so you should see these incentives as "game" points designed…
The logical conclusion is that they (the models) will eventually be linked to crypto payments though. This is where Lightning becomes important... Edit: To clarify, I'm not suggesting linking these Petal "tokens" to any payment system. I'm talking about, in general, calls to clusters of machine learning models, decentralized or not, will likely use crypto payments because it gives you auth and a means of payment. I d…
I mean, I can sell you Eve or Runescape currency but we don't need any crypto to execute on it. "Gold sellers" existed well before crypto.