Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Darkbloom – Private inference on idle Macs

darkbloom.dev

Post reply on HN
Print view View original

101–110 of 271 posts

Re: Darkbloom – Private inference on idle Macs

#101

Post by mike_hearn »

dinobones wrote: post #48
nl wrote: post #38

Earlier quoted context omitted.

Good point! But they argue that: > PT_DENY_ATTACH (ptrace constant 31): Invoked at process startup before any sensitive data is loaded. Instructs the macOS kernel to permanently deny all ptracerequests against this process, including from root. This blocks lldb, dtrace, and Instruments. > Hardened Runtime: The binary is code-signed with hardened runtime options and explicitly without the com.apple.security.get-task-a…

Couldn't someone just uhh... patch their macOS/kernel, mock these things out, then behold, you can now access all the data? If it's not running fully end to end in some secure enclave, then it's always just a best effort thing. Good marketing though.

Right.

Apple is perfectly capable of doing remote attestation properly. iOS has DCAppAttest which does everything needed. Unfortunately, it's never been brought to macOS, as far as I know. Maybe this MDM hack is a back door to get RA capabilities, if so it'd certainly be intriguing, but if not as far as I know there's no way to get a Mac to cough up a cryptographic assertion that it's running a genuine macOS kernel/boot firmware/disk image/kernel args, etc.

It's a pity because there's a lot of unique and interesting apps that'd become possible if Apple did this. Darkbloom is just one example of what's possible. It'd be a huge boon to decentralization efforts if Apple activated this, and all the pipework is laid already so it's really a pity they don't go the extra mile here.

Re: Darkbloom – Private inference on idle Macs

#102

Post by jorvi »

mavamaarten wrote: post #36
pants2 wrote: post #17

Cool idea. Just some back-of-the-envelope math here (not trusting what's on their site): My M5 Pro can generate 130 tok/s (4 streams) on Gemma 4 26B. Darkbloom's pricing is $0.20 per Mtok output. That's about $2.24/day or $67/mo revenue if it's fully utilized 24/7. Now assuming 50W sustained load, that's about 36 kWh/mo, at ~$.25/kWh approx. $9/mo in costs. Could be good for lunch money every once in a while! Around…

Well. Running your machine to do inference will utilize more than 50W sustained load, I'd say more than double that. Plus electricity is more expensive here (but granted, I do have solar panels). Plus don't forget to factor in that your hardware will age faster. I'd say it's not worth it. But the idea is cool.

Your hardware will age slower if you have consistent load.

Thermal stress from bursty workloads is much more of a wearing problem than electromigration. If you can consistently keep the SoC at a specific temperature, it'll last much longer.

This is also why it was very ironic that crypto miner GPUs would get sold at massive discounts. Everyone assumed that they had been ran ragged, but a proper miner would have undervolted the card and ran it at consistent utilization, meaning the card would be in better condition than a secondhand gamer GPU that would have constantly been shifting between 1% to 80% utilization, or rather, 30°C to 75°C

Re: Darkbloom – Private inference on idle Macs

#103

Post by mike_hearn »

ramoz wrote: post #29

Unfortunately, verifiable privacy is not physically possible on MacBooks of today. Don't let a nice presentation fool you. Apple Silicon has a Secure Enclave, but not a public SGX/TDX/SEV-style enclave for arbitrary code, so these claims are about OS hardening, not verifiable confidential execution. It would be nice if it were possible. There's a lot of cool innovations possible beyond privacy.

I wrote a whole SDK for using SGX, it's cool tech. But in theory on Apple platforms you can get a long way without it. iOS already offers this capability and it works OK.

macOS has a strong enough security architecture that something like Darkbloom would have at least some credibility if there was a way to remotely attest a Mac's boot sequence and TCC configuration combined with key-to-DR binding. The OS sandbox can keep apps properly separated if the kernel is correct and unhacked. And Apple's systems are full of mitigations and roadblocks to simple exploitation. Would it be as good as a consumer SGX enclave? Not architecturally, but the usability is higher.

Re: Darkbloom – Private inference on idle Macs

#104

Post by yard2010 »

tgma wrote: post #88
splittydev wrote: post #87

Earlier quoted context omitted.

I kind of see your point, but I also kind of don't. Sure, it would be great if you'd immediately get hammered with hundreds of requests and start make money quickly. It would also be great if it was a bit more transparent, and you could see more stats (what counts as "idle"? Is my machine currently eligible to serve models?). But it's still very new, I'd say give it some time and let's see how it goes. If you have it…

Well I already made the Ctrl+C decision. Yours may have been different, but I suppose only one of us installed it, and that one counts.

I went with the ctrl z approach.

Re: Darkbloom – Private inference on idle Macs

#105

Post by NiloCK »

znnajdla wrote: post #53
kennywinker wrote: post #8

I have a hard time believing their numbers. If you can pay off a mac mini in 2-4 months, and make $1-2k profit every month after that, why wouldn’t their business model just be buying mac minis?

The numbers are obviously high, because if this takes off then the price for inference will also drop. But I still think it’s a solid economic model that benefits low income countries the most. In Ukraine, for example, I know people who live on $200/month. A couple Mac Minis could feed a family in many places. As a business owner, I can think of multiple reasons why a decentralized network is better for me as a busin…

On the latency point - your requests are still going through the coordinator of the system here. So on average strictly worse than a large provider.

You - Darkbloom - Operator - Darkbloom - you, vs

You - Provider - you

---

On the censorship point - this is an interesting risk surface for operators. If people are drawn my decentralized model provisioning for its lax censorship, I'm pretty sure they're using it to generate things that I don't want to be liable for.

If anything, I could imagine dumber and stricter brand-safety style censorship on operator machines.

Re: Darkbloom – Private inference on idle Macs

#106

Post by yard2010 »

znnajdla wrote: post #53
kennywinker wrote: post #8

I have a hard time believing their numbers. If you can pay off a mac mini in 2-4 months, and make $1-2k profit every month after that, why wouldn’t their business model just be buying mac minis?

The numbers are obviously high, because if this takes off then the price for inference will also drop. But I still think it’s a solid economic model that benefits low income countries the most. In Ukraine, for example, I know people who live on $200/month. A couple Mac Minis could feed a family in many places. As a business owner, I can think of multiple reasons why a decentralized network is better for me as a busin…

It's quite funny thinking about a chimpanzee seeing a lot of bananas thinking this could feed my family and then same with humans only with Mac Minis

Re: Darkbloom – Private inference on idle Macs

#107

Post by mirashii »

gleenn wrote: post #99

You have to install their MDM device management software on your computer. Basically that computer is theirs now. So don't plan on just handing over your laptop temporarily unless you don't mind some company completely owning your box. Still might be a validate use for people with slightly old laptops lying around, but beware trying to share this computer with your daily activities if you e.g. use a bank on a browser…

MDMs on macOS are permissioned via AccessRights, and you can verify that their permission set is fairly minimal and does not allow what you've described here (bits 0, 4, 10).

That said, their privacy posture at the cornerstone of their claims is snake oil and has gaping holes in it, so I still wouldn't trust it, but it's worth being accurate about how exactly they're messing up.

Re: Darkbloom – Private inference on idle Macs

#109

Post by [deleted] »

saagarjha wrote: post #83
btown wrote: post #5

Earlier quoted context omitted.

From the paper: https://github.com/Layr-Labs/d-inference/blob/master/papers/... > Apple’s attestation servers will only generate the FreshnessCode for a genuine device that checks in via APNs. A software-only adversary cannot forge the MDA certificate chain (Assumption 3). Com- bined with SIP enforcement (preventing binary replace- ment) and Secure Boot (preventing bootloader tampering), this provides strong evidence…

I am not entirely sure they understand that System Integrity Protection and Secure Boot can be turned off.

[deleted]

Re: Darkbloom – Private inference on idle Macs

#110

Post by NiloCK »

Interesting to see an offering with this heritage [1] proposing flat earnings rates for inference operators here, rather than trying to sell a dynamic marketplace where operators compete on price in real-time.

Right now the dashboards show 78 providers online, but someone in-thread here said that they spun one up and got no requests. Surely someone would be willing to beat the posted rate and swallow up the demand?

I expect this is a migration target, but a tactical omission from V1 comms both for legitimate legibility reasons (I can sell x for y is easier to parse than 'I can participate in a marketplace') and slightly illegitimate legibility reasons (obscuring likely future price collapse).

Still - neat project that I hope does well.

[1] Layer Labs, formerly EigenLayer, is company built around a protocol to abstract and recycle economic security guarantees from Ethereum proof of stake.

Post reply on HN