Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Darkbloom – Private inference on idle Macs

darkbloom.dev

Post reply on HN
Print view View original

111–120 of 271 posts

Re: Darkbloom – Private inference on idle Macs

#111

Post by mike_hearn »

mirashii wrote: post #107
gleenn wrote: post #99

You have to install their MDM device management software on your computer. Basically that computer is theirs now. So don't plan on just handing over your laptop temporarily unless you don't mind some company completely owning your box. Still might be a validate use for people with slightly old laptops lying around, but beware trying to share this computer with your daily activities if you e.g. use a bank on a browser…

MDMs on macOS are permissioned via AccessRights, and you can verify that their permission set is fairly minimal and does not allow what you've described here (bits 0, 4, 10). That said, their privacy posture at the cornerstone of their claims is snake oil and has gaping holes in it, so I still wouldn't trust it, but it's worth being accurate about how exactly they're messing up.

Edit: deleted post. I see your other post now.

You are right - the "nonce binding" the paper uses doesn't seem convincing. The missing link is that Apple's attestation doesn't bind app generated keys to a designated requirement, which would be required to create a full remote attestation.

Re: Darkbloom – Private inference on idle Macs

#113

Post by [deleted] »

gleenn wrote: post #99

You have to install their MDM device management software on your computer. Basically that computer is theirs now. So don't plan on just handing over your laptop temporarily unless you don't mind some company completely owning your box. Still might be a validate use for people with slightly old laptops lying around, but beware trying to share this computer with your daily activities if you e.g. use a bank on a browser…

[deleted]

Re: Darkbloom – Private inference on idle Macs

#114

Post by mirashii »

mike_hearn wrote: post #111
mirashii wrote: post #107

Earlier quoted context omitted.

MDMs on macOS are permissioned via AccessRights, and you can verify that their permission set is fairly minimal and does not allow what you've described here (bits 0, 4, 10). That said, their privacy posture at the cornerstone of their claims is snake oil and has gaping holes in it, so I still wouldn't trust it, but it's worth being accurate about how exactly they're messing up.

Edit: deleted post. I see your other post now. You are right - the "nonce binding" the paper uses doesn't seem convincing. The missing link is that Apple's attestation doesn't bind app generated keys to a designated requirement, which would be required to create a full remote attestation.

> If you can prove a public key is generated by the SEP of a machine running with all Apple's security systems enabled, then you can trivially extend that to confidential computing because the macOS security architecture allows apps to block external inspection even by the root user.

It only effectively allows this for applications that are in the set of things covered by SIP, but not for any third-party application. There's nothing that will allow you to attest that arbitrary third-party code is running some specific version without being tampered with, you can only attest that the base OS/kernel have not been tampered with. In their specific case, they attempt to patch over that by taking the hash of the binary, but you can simply patch it before it starts.

To do this properly requires a TEE to be available to third-party code for attestation. That's not a thing on macOS today.

Re: Darkbloom – Private inference on idle Macs

#115

Post by mike_hearn »

mirashii wrote: post #114
mike_hearn wrote: post #111

Earlier quoted context omitted.

Edit: deleted post. I see your other post now. You are right - the "nonce binding" the paper uses doesn't seem convincing. The missing link is that Apple's attestation doesn't bind app generated keys to a designated requirement, which would be required to create a full remote attestation.

> If you can prove a public key is generated by the SEP of a machine running with all Apple's security systems enabled, then you can trivially extend that to confidential computing because the macOS security architecture allows apps to block external inspection even by the root user. It only effectively allows this for applications that are in the set of things covered by SIP, but not for any third-party application.…

I wiped my post because you are right. I don't think it needs a full SGX-style TEE. What's missing is a link to designated requirements. Abusing a nonce field doesn't seem to work, or if it does I can't figure out how. The MDM/MDA infrastructure would need to be able to include:

    public key from SEP -> designated requirement of owning app binary
The macOS KeyStore infrastructure does track this which is why I thought it'd work. But the paper doesn't mention being able to get this data server side anywhere. Instead there's this nonce hack.

It's odd that the paper considers so many angles including things like RDMA over Thunderbolt, but not the binding between platform key and app key.

Reading the paper again carefully I get the feeling the author knows or believes something that isn't fully elaborated in the text. He recognizes that this linkage problem exists, proposes a solution and offers a security argument for it. I just can't understand the argument. It appears APNS plays a role (apple push notification service) and maybe this is where app binding happens but the author seems to assume a fluency in Apple infrastructure that I currently lack.

Re: Darkbloom – Private inference on idle Macs

#116

Post by Jn2G3Np8 »

Love the concept, with some similarity to folding@home, though more personal gain.

But trying it out it still needs work, I couldn't download a model successfully (and their list of nodes at https://console.darkbloom.dev/providers suggests this is typical).

And as a cursory user, it took me some digging to find out that to cash out you need a Solana address (providers > earnings).

Re: Darkbloom – Private inference on idle Macs

#117

Post by mike_hearn »

nl wrote: post #6

They use the TEE to check that the model and code is untampered with. That's a good, valid approach and should work (I've done similar things on AWS with their TEE) The key question here is how they avoid the outside computer being able to view the memory of the internal process: > An in-process inference design that embeds the in- ference engine directly in a hardened process, elimi- nating all inter-process communi…

Apple Silicon systems have unified memory between CPU and GPU. The hypervisor page table trick is thus claimed to protect GPU memory from RDMA.

Re: Darkbloom – Private inference on idle Macs

#118

Post by znnajdla »

kennywinker wrote: post #63
znnajdla wrote: post #53

Earlier quoted context omitted.

The numbers are obviously high, because if this takes off then the price for inference will also drop. But I still think it’s a solid economic model that benefits low income countries the most. In Ukraine, for example, I know people who live on $200/month. A couple Mac Minis could feed a family in many places. As a business owner, I can think of multiple reasons why a decentralized network is better for me as a busin…

How many of those people who could live off $200USD/month can afford or already have a mac mini in the house?

They already have an iPhone. They could save up or borrow for a Mac Mini if they had to. Some of those people I know who live on $200/month have $30k in the bank.

Re: Darkbloom – Private inference on idle Macs

#119

Post by mirashii »

mike_hearn wrote: post #115
mirashii wrote: post #114

Earlier quoted context omitted.

> If you can prove a public key is generated by the SEP of a machine running with all Apple's security systems enabled, then you can trivially extend that to confidential computing because the macOS security architecture allows apps to block external inspection even by the root user. It only effectively allows this for applications that are in the set of things covered by SIP, but not for any third-party application.…

I wiped my post because you are right. I don't think it needs a full SGX-style TEE. What's missing is a link to designated requirements. Abusing a nonce field doesn't seem to work, or if it does I can't figure out how. The MDM/MDA infrastructure would need to be able to include: public key from SEP -> designated requirement of owning app binary The macOS KeyStore infrastructure does track this which is why I thought…

I can buy the idea that if you can have the MDM infrastructure attest the code signing identity through the designated requirements, that you can probably come pretty close, but I'm still not quite sure you get there with root on macOS (and I suspect that this is part of why DCAppAttest hasn't made it to macOS yet).

Certainly, it still doesn't get you there with their current implementation, as the attempts at blocking the debugger like PT_DENY_ATTACH are runtime syscalls, so you've got a race window where you can attach still. Maybe it gets you there with hardened runtime? I'd have to think a bit harder on that.

Re: Darkbloom – Private inference on idle Macs

#120

Post by znnajdla »

ramoz wrote: post #57
znnajdla wrote: post #55

Earlier quoted context omitted.

As if you get privacy with the inference providers available today? I have more trust in a randomly selected machine on a decentralized network not being compromised than in a centralized provider like OpenAI pinky promising not to read your chats.

Inference providers don't claim private inference. However, they must uphold certain security and legal compliances. You have no guarantees over any random connected laptop connected across the world.

I would say the chances of OpenAI itself getting hacked and your secrets in logs getting leaked are about the same or less as the chances of a randomly selected machine on a decentralized network being reverse-engineered by a determined hacker. There's no risk-free option, every provider comes with risks. If you care about infosec you have to do frequent secret rotation anyway.
Post reply on HN