Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Darkbloom – Private inference on idle Macs

darkbloom.dev

Post reply on HN
Print view View original

11–20 of 271 posts

Re: Darkbloom – Private inference on idle Macs

#11

Post by rvz »

chaoz_ wrote: post #9

That solution actually makes great sense. So Apple won in some strange way again? Guess there are limitations on size of the models, but if top-tier models will getting democratized I don’t see a reason not to use this API. The only thing that comes to me is data privacy concerns. I think batch-evals for non-sensitive data has great PMF here.

Yes. They never needed to participate in the AI race to zero.

Because they were already at the finish line with Apple Silicon.

> I don’t see a reason not to use this API. The only thing that comes to me is data privacy concerns.

The whole inference is end-to-end encrypted so none of the nodes can see the prompts or the messages.

Re: Darkbloom – Private inference on idle Macs

#14

Post by chaoz_ »

rvz wrote: post #11
chaoz_ wrote: post #9

That solution actually makes great sense. So Apple won in some strange way again? Guess there are limitations on size of the models, but if top-tier models will getting democratized I don’t see a reason not to use this API. The only thing that comes to me is data privacy concerns. I think batch-evals for non-sensitive data has great PMF here.

Yes. They never needed to participate in the AI race to zero. Because they were already at the finish line with Apple Silicon. > I don’t see a reason not to use this API. The only thing that comes to me is data privacy concerns. The whole inference is end-to-end encrypted so none of the nodes can see the prompts or the messages.

Fun question: can some (part of it) be a crypto token that I can buy? :))

That would finally be a crypto thing which is backed by value I believe in.

Re: Darkbloom – Private inference on idle Macs

#16

Post by chaoz_ »

kennywinker wrote: post #8

I have a hard time believing their numbers. If you can pay off a mac mini in 2-4 months, and make $1-2k profit every month after that, why wouldn’t their business model just be buying mac minis?

Solid q. I think the part of it is that it’s really easy to attract some “mass” (capital) of users, as there are definitely quite a few of idle Macs in the world.

Non-VC play (not required until you can raise on your own terms!) and clear differentiation.

If you want to go full-business-evaluation, I would be more worried about someone else implementing same thing with more commission (imo 95% and first to market is good enough).

Re: Darkbloom – Private inference on idle Macs

#17

Post by pants2 »

Cool idea. Just some back-of-the-envelope math here (not trusting what's on their site):

My M5 Pro can generate 130 tok/s (4 streams) on Gemma 4 26B. Darkbloom's pricing is $0.20 per Mtok output.

That's about $2.24/day or $67/mo revenue if it's fully utilized 24/7.

Now assuming 50W sustained load, that's about 36 kWh/mo, at ~$.25/kWh approx. $9/mo in costs.

Could be good for lunch money every once in a while! Around $700/yr.

Re: Darkbloom – Private inference on idle Macs

#18

Post by kennywinker »

gleenn wrote: post #15
kennywinker wrote: post #8

I have a hard time believing their numbers. If you can pay off a mac mini in 2-4 months, and make $1-2k profit every month after that, why wouldn’t their business model just be buying mac minis?

Power and racking are difficult and expensive?

How difficult? Is running 1000 minis worth $1,000,000/month of effort? I feel like it is.

Re: Darkbloom – Private inference on idle Macs

#20

Post by flockonus »

nl wrote: post #6

They use the TEE to check that the model and code is untampered with. That's a good, valid approach and should work (I've done similar things on AWS with their TEE) The key question here is how they avoid the outside computer being able to view the memory of the internal process: > An in-process inference design that embeds the in- ference engine directly in a hardened process, elimi- nating all inter-process communi…

While they do make this argument, realistically anyone sending their prompt/data to an external server should assume there will be some level of retention.

And more so in particular, anyone using Darkbloom with commercial intents should only really send non-sensitive data (no tokens, customer data, ...) I'd say only classification tasks, imagine generation, etc.

Post reply on HN