Show HN: Lumabri – Run Moe Models on a P2P Swarm with Colibri
1–10 of 25 posts
Re: Show HN: Lumabri – Run Moe Models on a P2P Swarm with Colibri
#2great work
Re: Show HN: Lumabri – Run Moe Models on a P2P Swarm with Colibri
#3great work
Thanks for the support!
Re: Show HN: Lumabri – Run Moe Models on a P2P Swarm with Colibri
#4P2P inference sounds really nice and it would bring us back to the 2000s culture if not for the fact inference is already so cheap. Even so it is an insurance policy if the cloud providers or governments have ideas of further censoring and monitoring chats.
Re: Show HN: Lumabri – Run Moe Models on a P2P Swarm with Colibri
#5Wasn't a p2p ai model how skynet got started?
J/k, this looks cool :)
Re: Show HN: Lumabri – Run Moe Models on a P2P Swarm with Colibri
#6Cool idea. How do you handle temperature in the verification?
Re: Show HN: Lumabri – Run Moe Models on a P2P Swarm with Colibri
#7[dead]
Re: Show HN: Lumabri – Run Moe Models on a P2P Swarm with Colibri
#8Cool idea. How do you handle temperature in the verification?
Ho thanks for the comment.
Verification does not depend on temperature.
Expert execution is deterministic (pure matmul).
LUMABRI_VERIFY=N re-runs N% of the calls on a second replica and requires byte-identical output.
Temperature (and sampling) happens only on the chatter, after the experts return their activations.
So it can be any value (0, 0.7, 1.2…) without affecting the verification contract.
Re: Show HN: Lumabri – Run Moe Models on a P2P Swarm with Colibri
#9I am sorry I did not understand all of it. But, would this allow running large MoE LLMs on a local network with experts spread out over multiple cheaper GPUs (or even CPUs)? This would perhaps be more useful than over the Internet, within offices for example.
Re: Show HN: Lumabri – Run Moe Models on a P2P Swarm with Colibri
#10What if one wishes to use various busybox nodes within the house?
All the iot devices contributing to matmul but within a LAN?