Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Darkbloom – Private inference on idle Macs

darkbloom.dev

Post reply on HN
Print view View original

251–260 of 271 posts

Re: Darkbloom – Private inference on idle Macs

#251

Post by willquack »

alexpotato wrote: post #165

Wasn't there an idea about 15 years ago where you would open your browser, go to a webpage and that page would have a JavaScript based client that would run distributed workloads? I believe the idea was that people could submit big workloads, the server would slice them up and then have the clients download and run a small slice. You as the computer owner would then get some payout. Intersting to see this coming back…

I used to work at Distributive (formerly "Kings Distributed Systems") on its DCP compute platform" which is entirely what you're describing. You can deploy a JS/WASM based workload, and it will be "sliced" and served to browser-based compute nodes. With WebGPU you can sort of have inference executing in the browser too. Incredible people there with an awesome project

I added Python execution support via Pyodide (cpython compiled to wasm) and worked on a bunch of other random stuff like WebLLM inferencing during my time there.

Apart from Distributive, there's also the "Golem network", "Salad", "Koii" and various other similar projects.

---

I'm not sure if I'm convinced by the "Uber for compute" use case with compute buyers and compute workers (sellers), but if you're a university and you have 1000 Windows machines across all your computer labs, it'd be nice to leverage that compute for running research or something idk - especially with the price of ram / cloud offerings these days...

Re: Darkbloom – Private inference on idle Macs

#252

Post by dnnddidiej »

CTDOCodebases wrote: post #208
dnnddidiej wrote: post #65

Earlier quoted context omitted.

It is too good to be true. When you see it is making more than a claude code subscription for fuck all work per day. Prolly gonna make $50 a year tops.

Or like anything else it will be too good to be true at the very beginning but then once people hear about it and it gets popular supply overtakes demand and the mac minis go back to being idle most of the day. When YouTubers start making videos about it you know it's too late.

The question is what is the hassle of running it plus wear and tear. Max price will tend to that. It is not like crypto where there is capital investment in a rig that can do nothing else. People are using their existing laptop. So I reckon 20-50 a year max per laptop.

Re: Darkbloom – Private inference on idle Macs

#253

Post by splintercell »

mbesto wrote: post #192
eigengajesh wrote: post #151

Earlier quoted context omitted.

The numbers are optimistically legit -- it's calculated based purely considering we have demand for all machines at all times. We don't have that right now, but fairly optimistic that people will do it. That's why we don't recommend purchasing a new machine. Existing machine is no cost for you to run this. Electricity is one cost, but it will get paid off from every request it receives. Electricity is only deducted w…

> That's why we don't recommend purchasing a new machine. Existing machine is no cost for you to run this. You misunderstood. If the ROI is there, there is enough capital in existence for you to accelerate your profit. So why even deal with the complexity of renting people's hardware when you can do it yourself?

No, what he's saying is that he expects this to be the ROI in the future because his product is so good.

Re: Darkbloom – Private inference on idle Macs

#254

Post by MithrilTuxedo »

BuildTheRobots wrote: post #217
eigengajesh wrote: post #151

Earlier quoted context omitted.

The numbers are optimistically legit -- it's calculated based purely considering we have demand for all machines at all times. We don't have that right now, but fairly optimistic that people will do it. That's why we don't recommend purchasing a new machine. Existing machine is no cost for you to run this. Electricity is one cost, but it will get paid off from every request it receives. Electricity is only deducted w…

I don't worry about bandwidth or constant CPU use, but the one thing that will kill my mac is burning out the SSD. The calculator gives numbers for nearly everything, but I can't obviously see how much space it needs for model storage or how many writes of temp files I should expect if I'm running flat out.

I assumed you'd want to run one immutable model that can fit in memory without any temp files.

Re: Darkbloom – Private inference on idle Macs

#255

Post by neurostimulant »

sleepybrett wrote: post #230
BuildTheRobots wrote: post #217

Earlier quoted context omitted.

I don't worry about bandwidth or constant CPU use, but the one thing that will kill my mac is burning out the SSD. The calculator gives numbers for nearly everything, but I can't obviously see how much space it needs for model storage or how many writes of temp files I should expect if I'm running flat out.

put that stuff on an external disk perhaps, it will eventually crater, but it's easier to replace than macbook internal storage (how are they doing mac minis these days?)

Isn't high memory pressure will result in high memory paging which will wear down the internal ssd?

Re: Darkbloom – Private inference on idle Macs

#256

Post by utopiah »

TuringNYC wrote: post #206
utopiah wrote: post #96

Earlier quoted context omitted.

Seems like so much more work than "just" paying for https://huggingface.co or whichever other neocloud who already did all the setup for you and just waits for your credit card per minute/seconds/token.

It is much more work because for many workloads you have geographic ringfencing and cannot send it out to the cloud

Doubt this kind of workloads would agree to send data then to a cloud of randos devices, precisely when cloud providers to certify they aren't looking at clients data (Customer-managed encryption keys, CMEK).

Re: Darkbloom – Private inference on idle Macs

#257

Post by saagarjha »

btown wrote: post #191
saagarjha wrote: post #83

Earlier quoted context omitted.

I am not entirely sure they understand that System Integrity Protection and Secure Boot can be turned off.

My understanding from the paper is that doing so should cause certain things in Apple's hardware security enclaves to break a signing chain, and a server-side MDM system integrated with Apple servers can detect this. But I'm not familiar with the underlying technology, so not sure if underlying assumptions are incorrect.

AFAIK that just ensures the SEP is present but perhaps they are signing the boot state now

Re: Darkbloom – Private inference on idle Macs

#258

Post by hamiltont »

tgma wrote: post #80
splittydev wrote: post #78

Earlier quoted context omitted.

They released this like a day ago, I'm not surprised that there's not enough demand right now. Give it some time to take off

You'd think to bootstrap a marketplace you'd spend your own money to feed fake requests (or perhaps allow free chat so that they induce requests). Still, absolute zero is an unacceptable number. Had this running for more than an hour.

> We're not taking funds from customers yet — we're personally paying for all the provider requests during this phase. Credit purchases are disabled.

This appears on their credit purchase page right now, but you have to email them to get credits (everyone starts with zero)

Re: Darkbloom – Private inference on idle Macs

#259

Post by p2detar »

keremimo wrote: post #123
gleenn wrote: post #99

You have to install their MDM device management software on your computer. Basically that computer is theirs now. So don't plan on just handing over your laptop temporarily unless you don't mind some company completely owning your box. Still might be a validate use for people with slightly old laptops lying around, but beware trying to share this computer with your daily activities if you e.g. use a bank on a browser…

MDM is the absolute deal breaker. No way in hell will I ever make my Macbook into an unsellable brick if they so decide to lock my computer with MDM. Even moreso, not for pennies/month

They can never have supervised permissions on your device unless your mac was wiped and registered in Apple Business Manager. As a BYOD, it cannot be locked.

Whether it makes sense to enroll a mac as a BYOD is another question.

Post reply on HN