Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Open Source Implementation of Apple's Private Compute Cloud

github.com

Post reply on HN
Print view View original

41–50 of 111 posts

Re: Open Source Implementation of Apple's Private Compute Cloud

#41

Post by saurik »

Lord-Jobo wrote: post #20
saurik wrote: post #18

Yes: "provably" private... unless you have $1000 for a logic analyzer and a steady hand to solder together a fake DDR module. https://news.ycombinator.com/item?id=45746753

well, also indefinite time and physical access.

Which is what the provider themselves have, by definition. The people who run these services are literally sitting next to the box day in and day out... this isn't "provably" anything. You can trust them not to take advantage of the fact that they own the hardware, and you can even claim it makes it ever so slightly harder for them to do so, but this isn't something where the word "provably" is anything other than a lie.

Re: Open Source Implementation of Apple's Private Compute Cloud

#42

Post by MangoToupe »

jmort wrote: post #34
MangoToupe wrote: post #31

@dang can we modify the title to acknowledge that it's specific to chatbots? The title reads like this is about generic compute, and the content is emphatically not about generic compute. I realize this is just bad branding by apple but it's still hella confusing.

It does work generically. Like Apple, we initially targeted inference, but it under the hood just an anonymous, attested HTTP server wrapper. The ComputeNode can run an arbitrary workload.

Interesting!

Re: Open Source Implementation of Apple's Private Compute Cloud

#44

Post by rimeice »

anon721656321 wrote: post #15
ryanMVP wrote: post #10

Reading the whitepaper, the inference provider still has the ability to access the prompt and response plaintext. This scheme does seem to guarantee that plaintext cannot be read for all other parties (e.g. the API router), and that the client's identity is hidden and cannot be associated with their request. Perhaps the precise privacy guarantees and allowances should be summarized in the readme. With that in mind, d…

at that point, it seems easier to run a slightly worse model locally. (or on a rented server)

Which is apples own approach until the compute requirements need them to run some compute on cloud.

Re: Open Source Implementation of Apple's Private Compute Cloud

#45

Post by sublimefire »

Lord-Jobo wrote: post #20
saurik wrote: post #18

Yes: "provably" private... unless you have $1000 for a logic analyzer and a steady hand to solder together a fake DDR module. https://news.ycombinator.com/item?id=45746753

well, also indefinite time and physical access.

If you trust the provider then it does not make it much better to use such architecture. If you do not then at least the execution should be inside a confidential system so that even soldering would not get you to data

Re: Open Source Implementation of Apple's Private Compute Cloud

#46

Post by anon721656321 »

saurik wrote: post #41
Lord-Jobo wrote: post #20

Earlier quoted context omitted.

well, also indefinite time and physical access.

Which is what the provider themselves have, by definition. The people who run these services are literally sitting next to the box day in and day out... this isn't "provably" anything. You can trust them not to take advantage of the fact that they own the hardware, and you can even claim it makes it ever so slightly harder for them to do so, but this isn't something where the word "provably" is anything other than a…

yeah, for a moment I was reading it as being a holomorphic encryption type setup, which I think is the only case where you can say 'provably private'.

It's better than nothing, I guess...

But if you placed the server at the NSA, and said "there is something on here that you really want, it's currently powered on and connected to the network, and the user is accessing it via ssh", it seems relatively straightforward for them to intercept and access.

Re: Open Source Implementation of Apple's Private Compute Cloud

#47

Post by jmort »

rasengan wrote: post #36
ryanMVP wrote: post #10

Reading the whitepaper, the inference provider still has the ability to access the prompt and response plaintext. This scheme does seem to guarantee that plaintext cannot be read for all other parties (e.g. the API router), and that the client's identity is hidden and cannot be associated with their request. Perhaps the precise privacy guarantees and allowances should be summarized in the readme. With that in mind, d…

We are introducing Verifiably Private AI [1] which actually solves all of the issues you mention. Everything across the entire chain is verifiably private (or in other words, transparent to the user in such a way they can verify what is running across the entire architecture). [1] https://ai.vp.net/

It should be able to support connecting via an OpenPCC client, then!

Re: Open Source Implementation of Apple's Private Compute Cloud

#49

Post by sublimefire »

Terretta wrote: post #11
ryanMVP wrote: post #10

Reading the whitepaper, the inference provider still has the ability to access the prompt and response plaintext. This scheme does seem to guarantee that plaintext cannot be read for all other parties (e.g. the API router), and that the client's identity is hidden and cannot be associated with their request. Perhaps the precise privacy guarantees and allowances should be summarized in the readme. With that in mind, d…

> the inference provider still has the ability to access the prompt and response plaintext Folks may underestimate the difficulty of providing compute that the provider “cannot”* access to reveal even at gunpoint. BYOK does cover most of it, but oh look, you brought me and my code your key, thanks… Apple's approach, and certain other systems such as AWS's Nitro Enclaves, aim at this last step of the problem: - https:…

Yes but at the end of the day you need to trust the cloud provider tools which expands the trust boundary from just hardware root of trust. Who is to guarantee they will not create a malicious tool update and push it then retract it? It is nowhere captured and you cannot prove it.

Re: Open Source Implementation of Apple's Private Compute Cloud

#50

Post by almostgotcaught »

7e wrote: post #23
Terretta wrote: post #11

Earlier quoted context omitted.

> the inference provider still has the ability to access the prompt and response plaintext Folks may underestimate the difficulty of providing compute that the provider “cannot”* access to reveal even at gunpoint. BYOK does cover most of it, but oh look, you brought me and my code your key, thanks… Apple's approach, and certain other systems such as AWS's Nitro Enclaves, aim at this last step of the problem: - https:…

At the end if the day, Nitro Enclaves are still “trust Amazon”, which is a poor guarantee. NVIDIA+AMD offers hardware backed enclave features for their GPUs which is the superior solution here.

be sure to let us know when you can run eg nginx on a GPU in said enclave.
Post reply on HN