Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Darkbloom – Private inference on idle Macs

darkbloom.dev

Post reply on HN
Print view View original

261–270 of 271 posts

Re: Darkbloom – Private inference on idle Macs

#261

Post by tgma »

elbac wrote: post #243
tgma wrote: post #50

I installed this so you don't have to. It did feel a bit quirky and not super polished. Fails to download the image model. The audio/tts model fails to load. In 15 minutes of serving Gemma, I got precisely zero actual inference requests, and a bunch of health checks and two attestations. At the moment they don't have enough sustained demand to justify the earning estimates.

I received the same error, but it was followed by this line in the logs, which might explain the lack of inference requests assume there is actual demand... WARN STT backend failed health check — model will NOT be advertised

I disabled the audio model (had to remove it, it was so buggy) and then started up with a text only model. Serving started without error. The system simply has no requests. The economics seem like a mirage anyway.

Re: Darkbloom – Private inference on idle Macs

#262

Post by flockonus »

joelthelion wrote: post #143
flockonus wrote: post #20

Earlier quoted context omitted.

While they do make this argument, realistically anyone sending their prompt/data to an external server should assume there will be some level of retention. And more so in particular, anyone using Darkbloom with commercial intents should only really send non-sensitive data (no tokens, customer data, ...) I'd say only classification tasks, imagine generation, etc.

There's a difference between trusting Anthropic and trusting random mac owners.

I know where my answer lies in that; but i don't claim to be an objective truth.

For example OpenAI has been caught sharing data with the gov. agencies.

Re: Darkbloom – Private inference on idle Macs

#263

Post by PiersonMarks »

MicBook56 wrote: post #164

I like the idea but it wont take off until Homomorphic Encryption for inference becomes a thing that's efficient and anyone can be a node.

This is what I was confused about like if they own their device they can see what's happening there until we solve this

Re: Darkbloom – Private inference on idle Macs

#265

Post by afcool83 »

frankfrank13 wrote: post #212

This is one of those ideas I think makes perfect sense, but requires so much operational change for the entire stack, that it would be very difficult to scale: - Convincing labs to run distributed, burst-y inference - Convincing people to run their Mac all day, hoping to make a little profit - Convincing users to trust a distributed network of un-trusted devices I had a similar idea, pre-AI, just for compute in gener…

...or convincing operators that jobs sent to their machines are legal, legitimate, and non-nefarious.

I could not find disclosure on their site about the guard-railing or safety-systems at the point the prompt is gathered from users which would intercept, log & prevent bad actors from inadvertently involving me in something illegal or immoral as an operator. Perhaps that disclosure exists and I just need to be linked to it; that would be welcome.

Re: Darkbloom – Private inference on idle Macs

#266

Post by TuringNYC »

utopiah wrote: post #256
TuringNYC wrote: post #206

Earlier quoted context omitted.

It is much more work because for many workloads you have geographic ringfencing and cannot send it out to the cloud

Doubt this kind of workloads would agree to send data then to a cloud of randos devices, precisely when cloud providers to certify they aren't looking at clients data (Customer-managed encryption keys, CMEK).

>> Doubt this kind of workloads would agree to send data then to a cloud of randos devices,

Totally agree, which is why i said "I'd love a way to do this locally -- pool all the PCs in our own office for in-office pools of compute."

Re: Darkbloom – Private inference on idle Macs

#267

Post by alexpotato »

willquack wrote: post #251
alexpotato wrote: post #165

Wasn't there an idea about 15 years ago where you would open your browser, go to a webpage and that page would have a JavaScript based client that would run distributed workloads? I believe the idea was that people could submit big workloads, the server would slice them up and then have the clients download and run a small slice. You as the computer owner would then get some payout. Intersting to see this coming back…

I used to work at Distributive (formerly "Kings Distributed Systems") on its DCP compute platform" which is entirely what you're describing. You can deploy a JS/WASM based workload, and it will be "sliced" and served to browser-based compute nodes. With WebGPU you can sort of have inference executing in the browser too. Incredible people there with an awesome project I added Python execution support via Pyodide (cpyt…

> but if you're a university and you have 1000 Windows machines across all your computer labs, it'd be nice to leverage that compute for running research or something idk - especially with the price of ram / cloud offerings these days...

This reminds me of the DevOps guy who made the developer laptops part of a Jenkins "swarm" under the thought that the machines were beefy and underutilized most of the time.

Re: Darkbloom – Private inference on idle Macs

#268

Post by rvz »

kennywinker wrote: post #47
rvz wrote: post #34

Earlier quoted context omitted.

Good. Another great non-speculative use-case for crypto and stablecoins.

Amazing! Let me see, doing the math r/n… carry the one, yup that makes the total number of non-speculative uses for crypto and stablecoin: 1 ;P

It always has been payments. x402 and Stripe Tempo makes the use case more than 1.

Re: Darkbloom – Private inference on idle Macs

#269

Post by Xx_crazy420_xX »

"Debugger attachment is blocked. Memory inspection is blocked." - reminds me old crackme challenges. Everything they mention can be bypassed, so determined person can start stealing data from the network. For me this is a killer of such distributed compute ideas, but who knows, maybe the non-enteprise users will be desperate enough for cheap compute to make this idea valid.
Post reply on HN