Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Private Cloud Compute: A new frontier for AI privacy in the cloud

security.apple.com

Post reply on HN
Print view View original

131–140 of 393 posts

Re: Private Cloud Compute: A new frontier for AI privacy in the cloud

#131

Post by asadotzler »

riscy wrote: post #47
SirensOfTitan wrote: post #36

What I'm most curious about here is if a state actor comes to Apple with a subpoena and compels them to release information on an individual, what would Apple be able to release? ... I suppose this is ultimately a question that will be tested sooner or later in the US.

I mean it was famously tested in 2015 after the San Bernardino attack. Apple didn’t back down [1] and later sued the company who sold the zero-day to the govt to unlock the phone [2]. [1] https://en.m.wikipedia.org/wiki/Apple%E2%80%93FBI_encryption... [2] https://www.washingtonpost.com/technology/2021/04/14/azimuth...

Also famously tested (and failed) much more recently. https://arstechnica.com/tech-policy/2023/12/apple-admits-to-...

Apple shills are the worst.

Re: Private Cloud Compute: A new frontier for AI privacy in the cloud

#132

Post by buzzerbetrayed »

zmmmmm wrote: post #129

Read through it all, it still comes down to "trust us". Apple can sign and authorise an update at any time that will backdoor it, and the government is the stroke of a pen away from forcing them to, all completely silently. I get that there's benefit to what they are doing. But the problem of selling a message of trust is you absolutely have to be 100% truthful about it, and them failing to be transparent that people…

Your argument is no different than what Apple could do to your iPhone. The fact that it happens on the server changes nothing. Apple could push a button and have your iPhone upload whatever they want to their servers. In other words, based on your argument, you shouldn't trust anything, including locally run AI. You're probably right, but it isn't practical.

Edit: The final couple tweets from the Matthew Green tweet thread posted in another comment sum it up well:

> Wrapping up on a more positive note: it’s worth keeping in mind that sometimes the perfect is the enemy of the really good.

> In practice the alternative to on-device is: ship private data to OpenAI or someplace sketchier, where who knows what might happen to it. And of course, keep in mind that super-spies aren’t your biggest adversary. For many people your biggest adversary is the company who sold you your device/software. This PCC system represents a real commitment by Apple not to “peek” at your data. That’s a big deal. In any case, this is the world we’re moving to. Your phone might seem to be in your pocket, but a part of it lives 2,000 miles away in a data center. As security folks we probably need to get used to that fact, and do the best we can to make sure all parts are secure.

Re: Private Cloud Compute: A new frontier for AI privacy in the cloud

#133

Post by a2128 »

CGamesPlay wrote: post #121

All of this is interesting, but how easy is this to circumvent? When Apple changes their mind for whatever reason, don't they just return a key to a fake PCC node, which would bypass all of their listed protections? Furthermore, what prevents Apple from doing this for specific users?

According to the article, it would be difficult to tie any request to a user:

> Target diffusion starts with the request metadata, which leaves out any personally identifiable information about the source device or user, and includes only limited contextual data about the request that’s required to enable routing to the appropriate model

If this is the case, I wonder how the authentication would work. Is it a security through obscurity sort of situation? Wouldn't it be possible for someone, through extensive reverse engineering, to write a client in Python that gives you a nice free chat API and Apple would be none the wiser?

Re: Private Cloud Compute: A new frontier for AI privacy in the cloud

#134

Post by visarga »

amiantos wrote: post #119
ein0p wrote: post #78

It is not possible for this to be fully private in the United States because the government not only can force Apple to open up the kimono, it can also forbid it to talk about it. There’s not really anything Apple can do to work around this “limitation”. Thank your “representative” for extending the PATRIOT Act when you get a chance.

Private Cloud Compute servers have no persistent storage so there would be nothing to see upon opening the kimono. You'd need some sort of government requested live wire tap thing to harvest the data out of the incoming requests, which might be a different situation. I'm, of course, just some dude on the internet, thinking up a counter-point to this concern, who knows if I am even remotely in the right ballpark.

mandatory 30 day retention policies or something like it

Re: Private Cloud Compute: A new frontier for AI privacy in the cloud

#135

Post by tharant »

rjeli wrote: post #115
wslh wrote: post #103

Earlier quoted context omitted.

Yes, the issue is that they are really slow.

ZKML is actually not horrible, probably only 100-1000x overhead atm. Unfortunately it doesn’t solve the problem, you would need FHE which has much higher overhead

FHE? I, a noob, assume that acronym maybe has something to do with homomorphic encryption?

Also, got any links for interesting ZKML papers/projects?

Re: Private Cloud Compute: A new frontier for AI privacy in the cloud

#136

Post by troad »

zmmmmm wrote: post #129

Read through it all, it still comes down to "trust us". Apple can sign and authorise an update at any time that will backdoor it, and the government is the stroke of a pen away from forcing them to, all completely silently. I get that there's benefit to what they are doing. But the problem of selling a message of trust is you absolutely have to be 100% truthful about it, and them failing to be transparent that people…

They already have root. Their software is closed source. There is absolutely nothing stopping them from uploading all of your data right now.

If you don't trust the people making your OS, your problems are much deeper than fretting about off-device AI processing.

Re: Private Cloud Compute: A new frontier for AI privacy in the cloud

#137

Post by al_borland »

gpm wrote: post #125
al_borland wrote: post #70

Earlier quoted context omitted.

Apple needs to differentiate itself, and they have chosen privacy as a way to do that, which I'm all for. The headlines around Microsoft's AI efforts have largely been a nightmare, with a ton of bad press. If the press around Apple's AI is all about how over the top they went with security and privacy, that will likely make people feel a little better about using it. I'm not a big user of OpenAI's stuff, but if I was…

I actually thought one notable thing in the presentation was that they spent all this time talking about their new private cloud compute architecture. And then showed that they have a prompt asking if you're ok sending the data to OpenAI. Presumably because despite OpenAI promising not to use your data (a promise apple relayed) OpenAI didn't buy into this new architecture.

Thank you for mentioning this. I thought I was going crazy, because I heard this too, but kept seeing comment after comment on other sites asking if a person could choose not to use OpenAI, or that it was happening magically in the background. The way I heard it, the user was in control.

I think this goes back to what Steve said in 2010.

https://youtube.com/watch?v=Ij-jlF98SzA

And yes, while the data might not be linked to the user and striped of sensitive data, I could see people not wanting something very personal things to go to OpenAI, even if there should be no link. For example, I wouldn’t want any of my pictures going to OpenAI unless I specifically say it is OK for a given image.

Re: Private Cloud Compute: A new frontier for AI privacy in the cloud

#138

Post by aalimov_ »

gpm wrote: post #75
SirensOfTitan wrote: post #36

What I'm most curious about here is if a state actor comes to Apple with a subpoena and compels them to release information on an individual, what would Apple be able to release? ... I suppose this is ultimately a question that will be tested sooner or later in the US.

Probably everything uploaded after the intercept is in place if you can convince a court to compel it. One option is to release a malicious software update, sign it, publish the signature on the public chain, and then simply not release the binaries until after whatever associated gag orders there are (if any) expire. Apple gave themselves a 90 day timeline for this before they'd even be in violation of their promise…

> One option is to release a malicious software update, sign it, publish the signature on the public chain,

In this option it would be Apple releasing a malicious software update?

> If they can still create new hardware, it seems likely whoever is making that hardware must still have access to the keys...

This option reads like the keys are stored in apple-keys.txt

> Both of these attacks are outside the "threat model" proposed, because they are broad compromises against the entire PCC infrastructure

They mentioned that the in-depth write up will be shared later, might they still address this concern in writing? Your wording makes you sound so certain, but this is just a broad overview. How are you so sure?

Re: Private Cloud Compute: A new frontier for AI privacy in the cloud

#139

Post by zie »

astrange wrote: post #91
zie wrote: post #60

Earlier quoted context omitted.

Well the options from China's perspective is: Come to the table and meet some/all of our demands or stop doing business here. Since Apple devices are now on the Chinese Governments poopy list, I assume Apple is only meeting some, not all of China's demands. I assume if Apple did everything the Chinese govt wanted, they wouldn't be on the poopy list. Personally I see being on the Chinese govt poopy list as an endorsem…

It's a silly oversimplification that nothing in China is ever allowed to have privacy ever. China has privacy/data protection laws just like other countries do. Even an authoritarian government doesn't want other random private actors getting to see everything.

I agree, but I was talking specifically about the govt.

The govt basically requires total access doesn't it? I mean every govt basically wants it, and the US has tried many times, but so far hasn't quite gotten complete access everywhere.

Re: Private Cloud Compute: A new frontier for AI privacy in the cloud

#140

Post by __MatrixMan__ »

troad wrote: post #136
zmmmmm wrote: post #129

Read through it all, it still comes down to "trust us". Apple can sign and authorise an update at any time that will backdoor it, and the government is the stroke of a pen away from forcing them to, all completely silently. I get that there's benefit to what they are doing. But the problem of selling a message of trust is you absolutely have to be 100% truthful about it, and them failing to be transparent that people…

They already have root. Their software is closed source. There is absolutely nothing stopping them from uploading all of your data right now. If you don't trust the people making your OS, your problems are much deeper than fretting about off-device AI processing.

That's true, but also it should be possible to make an OS that people can trust without trusting you, and as users we should encourage movement in that direction.
Post reply on HN