Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Darkbloom – Private inference on idle Macs

darkbloom.dev

Post reply on HN
Print view View original

81–90 of 271 posts

Re: Darkbloom – Private inference on idle Macs

#81

Post by saagarjha »

dinobones wrote: post #48
nl wrote: post #38

Earlier quoted context omitted.

Good point! But they argue that: > PT_DENY_ATTACH (ptrace constant 31): Invoked at process startup before any sensitive data is loaded. Instructs the macOS kernel to permanently deny all ptracerequests against this process, including from root. This blocks lldb, dtrace, and Instruments. > Hardened Runtime: The binary is code-signed with hardened runtime options and explicitly without the com.apple.security.get-task-a…

Couldn't someone just uhh... patch their macOS/kernel, mock these things out, then behold, you can now access all the data? If it's not running fully end to end in some secure enclave, then it's always just a best effort thing. Good marketing though.

Yes. Running attested workloads on macOS if you are not Apple is nontrivial.

Re: Darkbloom – Private inference on idle Macs

#82

Post by saagarjha »

nl wrote: post #38
ramoz wrote: post #21

Earlier quoted context omitted.

Macs do not have an accessible hardware TEE. Macs have secure enclaves.

Good point! But they argue that: > PT_DENY_ATTACH (ptrace constant 31): Invoked at process startup before any sensitive data is loaded. Instructs the macOS kernel to permanently deny all ptracerequests against this process, including from root. This blocks lldb, dtrace, and Instruments. > Hardened Runtime: The binary is code-signed with hardened runtime options and explicitly without the com.apple.security.get-task-a…

They quite frankly have no idea what they are talking about.

Re: Darkbloom – Private inference on idle Macs

#83

Post by saagarjha »

btown wrote: post #5
DeathArrow wrote: post #2

Why only Macs? If we think of all PCs and mobile phones running idle, the potential is much larger.

From the paper: https://github.com/Layr-Labs/d-inference/blob/master/papers/... > Apple’s attestation servers will only generate the FreshnessCode for a genuine device that checks in via APNs. A software-only adversary cannot forge the MDA certificate chain (Assumption 3). Com- bined with SIP enforcement (preventing binary replace- ment) and Secure Boot (preventing bootloader tampering), this provides strong evidence…

I am not entirely sure they understand that System Integrity Protection and Secure Boot can be turned off.

Re: Darkbloom – Private inference on idle Macs

#84

Post by woadwarrior01 »

I won't install some random untrusted binary off of some website. I downloaded it and did some cursory analysis instead.

Got the latest v0.3.8 version from the list here: https://api.darkbloom.dev/v1/releases/latest

Three binaries and a Python file: darkbloom (Rust)

eigeninference-enclave (Swift)

ffmpeg (from Homebrew, lol)

stt_server.py (a simple FastAPI speech-to-text server using mlx_audio).

The good parts: All three binaries are signed with a valid Apple Developer ID and have Hardened runtime enabled.

Bad parts: Binaries aren't notarized. Enrolls the device for remote MDM using micromdm. Downloads and installs a complete Python runtime from Cloudflare R2 (Supply chain risk). PT_DENY_ATTACH to make debugging harder. Collects device serial numbers.

TL;DR: No, not touching that.

Re: Darkbloom – Private inference on idle Macs

#85

Post by WatchDog »

I installed two models, but it just always reports:

    Available models (2):
    CohereLabs/cohere-transcribe-03-2026 (4.6 GB)
    flux_2_klein_9b_q8p.ckpt (20.2 GB)
    ...
    Advertising 0 model(s) (only loaded models)
Also the benchmark just doesn't work.

Interesting idea, but needs some work.

Re: Darkbloom – Private inference on idle Macs

#86

Post by mirashii »

nl wrote: post #38
ramoz wrote: post #21

Earlier quoted context omitted.

Macs do not have an accessible hardware TEE. Macs have secure enclaves.

Good point! But they argue that: > PT_DENY_ATTACH (ptrace constant 31): Invoked at process startup before any sensitive data is loaded. Instructs the macOS kernel to permanently deny all ptracerequests against this process, including from root. This blocks lldb, dtrace, and Instruments. > Hardened Runtime: The binary is code-signed with hardened runtime options and explicitly without the com.apple.security.get-task-a…

Looking at their paper at [1], there's a gaping hole: there's no actual way to verify the contents of the running binaries. The binary hash they include in their signatures is self-reported, and can be modified. That's simply game over.

[1] https://github.com/Layr-Labs/d-inference/blob/master/papers/...

Re: Darkbloom – Private inference on idle Macs

#87

Post by splittydev »

tgma wrote: post #80
splittydev wrote: post #78

Earlier quoted context omitted.

They released this like a day ago, I'm not surprised that there's not enough demand right now. Give it some time to take off

You'd think to bootstrap a marketplace you'd spend your own money to feed fake requests (or perhaps allow free chat so that they induce requests). Still, absolute zero is an unacceptable number. Had this running for more than an hour.

I kind of see your point, but I also kind of don't.

Sure, it would be great if you'd immediately get hammered with hundreds of requests and start make money quickly. It would also be great if it was a bit more transparent, and you could see more stats (what counts as "idle"? Is my machine currently eligible to serve models?). But it's still very new, I'd say give it some time and let's see how it goes.

If you have it running and you get zero requests, it uses close to zero power above what your computer uses anyway. It doesn't cost you anything to have it running, and if you get requests, you make money. Seems like an easy decision to me.

Re: Darkbloom – Private inference on idle Macs

#88

Post by tgma »

splittydev wrote: post #87
tgma wrote: post #80

Earlier quoted context omitted.

You'd think to bootstrap a marketplace you'd spend your own money to feed fake requests (or perhaps allow free chat so that they induce requests). Still, absolute zero is an unacceptable number. Had this running for more than an hour.

I kind of see your point, but I also kind of don't. Sure, it would be great if you'd immediately get hammered with hundreds of requests and start make money quickly. It would also be great if it was a bit more transparent, and you could see more stats (what counts as "idle"? Is my machine currently eligible to serve models?). But it's still very new, I'd say give it some time and let's see how it goes. If you have it…

Well I already made the Ctrl+C decision. Yours may have been different, but I suppose only one of us installed it, and that one counts.

Re: Darkbloom – Private inference on idle Macs

#90

Post by mirashii »

mirashii wrote: post #86
nl wrote: post #38

Earlier quoted context omitted.

Good point! But they argue that: > PT_DENY_ATTACH (ptrace constant 31): Invoked at process startup before any sensitive data is loaded. Instructs the macOS kernel to permanently deny all ptracerequests against this process, including from root. This blocks lldb, dtrace, and Instruments. > Hardened Runtime: The binary is code-signed with hardened runtime options and explicitly without the com.apple.security.get-task-a…

Looking at their paper at [1], there's a gaping hole: there's no actual way to verify the contents of the running binaries. The binary hash they include in their signatures is self-reported, and can be modified. That's simply game over. [1] https://github.com/Layr-Labs/d-inference/blob/master/papers/...

A note, as others have posted on this thread: I mention this as a concrete and trivial flaw in their whole strategy, but the issue is fundamental: there's no hardware enclave for third-party code available to do the type of attestation that would be necessary. Any software approach they develop will ultimately fall to that hole.
Post reply on HN