Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Darkbloom – Private inference on idle Macs

darkbloom.dev

Post reply on HN
Print view View original

31–40 of 271 posts

Re: Darkbloom – Private inference on idle Macs

#31

Post by MrDrMcCoy »

nl wrote: post #7
DeathArrow wrote: post #2

Why only Macs? If we think of all PCs and mobile phones running idle, the potential is much larger.

They use the Apple TEE which they claim also protects GPU memory (I wasn't aware of this). NVidia data center GPUs have a similar path, but not their consumer ones. Not sure about the NVidia Spark. It's possible AMD Strix Halo can do this, but unlikely for any other PC based GPU environments.

Epyc has that VM encrypted memory thing, which comes pretty close. It does raise an interesting question, though: would a PCIe card passed through to a VM be able to DMA access the memory of neighboring devices?

Re: Darkbloom – Private inference on idle Macs

#32

Post by dr_kiszonka »

"These are estimates only. We do not guarantee any specific utilization or earnings. Actual earnings depend on network demand, model popularity, your provider reputation score, and how many other providers are serving the same model.

When your Mac is idle (no inference requests), it consumes minimal power — you don't lose significant money waiting for requests. The electricity costs shown only apply during active inference.

Text models typically see the highest and most consistent demand. Image generation and transcription requests are bursty — high volume during peaks, quiet otherwise."

Re: Darkbloom – Private inference on idle Macs

#35

Post by pants2 »

chaoz_ wrote: post #28
pants2 wrote: post #17

Cool idea. Just some back-of-the-envelope math here (not trusting what's on their site): My M5 Pro can generate 130 tok/s (4 streams) on Gemma 4 26B. Darkbloom's pricing is $0.20 per Mtok output. That's about $2.24/day or $67/mo revenue if it's fully utilized 24/7. Now assuming 50W sustained load, that's about 36 kWh/mo, at ~$.25/kWh approx. $9/mo in costs. Could be good for lunch money every once in a while! Around…

Genuinely curious, is there any way to estimate amortization of Mac? I’d imagine 1 year of heavy usage would somehow affect its quality.

Yeah, only way to get there is assuming they're not giving prompt caching discounts while my laptop is getting prompt caching benefits, with very many large prompts. So yes I am skeptical of their numbers.

Re: Darkbloom – Private inference on idle Macs

#36

Post by mavamaarten »

pants2 wrote: post #17

Cool idea. Just some back-of-the-envelope math here (not trusting what's on their site): My M5 Pro can generate 130 tok/s (4 streams) on Gemma 4 26B. Darkbloom's pricing is $0.20 per Mtok output. That's about $2.24/day or $67/mo revenue if it's fully utilized 24/7. Now assuming 50W sustained load, that's about 36 kWh/mo, at ~$.25/kWh approx. $9/mo in costs. Could be good for lunch money every once in a while! Around…

Well. Running your machine to do inference will utilize more than 50W sustained load, I'd say more than double that. Plus electricity is more expensive here (but granted, I do have solar panels). Plus don't forget to factor in that your hardware will age faster.

I'd say it's not worth it. But the idea is cool.

Re: Darkbloom – Private inference on idle Macs

#37

Post by geon »

ramoz wrote: post #29

Unfortunately, verifiable privacy is not physically possible on MacBooks of today. Don't let a nice presentation fool you. Apple Silicon has a Secure Enclave, but not a public SGX/TDX/SEV-style enclave for arbitrary code, so these claims are about OS hardening, not verifiable confidential execution. It would be nice if it were possible. There's a lot of cool innovations possible beyond privacy.

Every hardware key will be broken if there is enough incentive to do so. Their claims read like pure hubris.

Re: Darkbloom – Private inference on idle Macs

#38

Post by nl »

ramoz wrote: post #21
nl wrote: post #6

They use the TEE to check that the model and code is untampered with. That's a good, valid approach and should work (I've done similar things on AWS with their TEE) The key question here is how they avoid the outside computer being able to view the memory of the internal process: > An in-process inference design that embeds the in- ference engine directly in a hardened process, elimi- nating all inter-process communi…

Macs do not have an accessible hardware TEE. Macs have secure enclaves.

Good point!

But they argue that:

> PT_DENY_ATTACH (ptrace constant 31): Invoked at process startup before any sensitive data is loaded. Instructs the macOS kernel to permanently deny all ptracerequests against this process, including from root. This blocks lldb, dtrace, and Instruments.

> Hardened Runtime: The binary is code-signed with hardened runtime options and explicitly without the com.apple.security.get-task-allow entitlement. The kernel denies task_for_pid() and mach_vm_read()from any external process.

> System Integrity Protection (SIP): Enforces both of the above at the kernel level. With SIP enabled, root cannot circumvent Hardened Runtime protections, load unsigned kernel extensions, or modify protected sys- tem binaries. Section 5.1 proves that SIP, once verified, is immutable for the process lifetime.

gives them memory protection.

To me that is surprising.

Re: Darkbloom – Private inference on idle Macs

#40

Post by runako »

kennywinker wrote: post #18
gleenn wrote: post #15

Earlier quoted context omitted.

Power and racking are difficult and expensive?

How difficult? Is running 1000 minis worth $1,000,000/month of effort? I feel like it is.

There are many people who do not have ready access to a million dollars to purchase said Mac minis, much less the operating capital to rack & operate them.

Very smart play to build a platform, get scale, and prove out the software. Then either add a small network fee (this could be on money movement on/off platform), add a higher tier of service for money, and/or just use the proof points to go get access to capital and become an operator in your own pool.

Post reply on HN