Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Private Cloud Compute: A new frontier for AI privacy in the cloud

security.apple.com

Post reply on HN
Print view View original

251–260 of 393 posts

Re: Private Cloud Compute: A new frontier for AI privacy in the cloud

#251

Post by nl »

hexage1814 wrote: post #168

The thing with cloud and with anything related to it, anything that connects to the internet somehow... is that, unless it's open source and the servers decentralized, you are always trusting SOMEONE. Sure, Apple might make their best to ensure nobody – but them – have access to your data... but Apple controls all the end points. It controls the updates your iPhone receives, it controls the servers where this happens…

This isn't right.

If you trust math you can prove the software is what they say it is.

Yes it is work to do this, but this is a big step forward.

Re: Private Cloud Compute: A new frontier for AI privacy in the cloud

#252

Post by KaiserPro »

theshrike79 wrote: post #213
verisimi wrote: post #181

Earlier quoted context omitted.

It's completely fair, because regardless of third party audits, chips, etc, there are backdoors right along the line, that are going to provide Apple and the government with secret legal access to your data. They can simply go to a secret court, receive a secret judgment, and be authorised to secretly view your data. Does anyone really think this is not already the case? There is no transparency. A licensed third par…

The best protection against "secret orders" is to use mathematics. Build your system so that it can't be decrypted, don't log anything etc. Mullvad has been doing this with VPNs and law enforcement has tested it - there's nothing for them to get. Same has been proven with Apple not allowing FBI to open an iPhone, because it'd set a precedent. Future iPhone versions were made so that it's literally impossible for even…

> Build your system so that it can't be decrypted

Now you can't debug anything.

> Mullvad has been doing this with VPNs

Mullvad do not need to store any data at all. Infact any data that they store is a risk. Minimising the data stored minimises their risk. The only thing they need to store is keys.

Look, if you want to ask an AI service if this photo has a dog in, thats simple and requires no state other than the photo. If you want to ask it does it have my dog in, thats a whole 'nother kettle of fish. How do you communicate the descriptors that describe your dog? how do you generate them? on device? that'll drain your battery in a very short order.

> Apple not allowing FBI to open an iPhone, because it'd set a precedent

Because they didn't follow process.

> Future iPhone versions were made so that it's literally impossible for even Apple to open a locked iPhone.

They don't need to, just hack the icloud backup. plus its not impossible, its just difficult. If you own the key authority then its less hard.

Re: Private Cloud Compute: A new frontier for AI privacy in the cloud

#253

Post by filleokus »

a2128 wrote: post #133
CGamesPlay wrote: post #121

All of this is interesting, but how easy is this to circumvent? When Apple changes their mind for whatever reason, don't they just return a key to a fake PCC node, which would bypass all of their listed protections? Furthermore, what prevents Apple from doing this for specific users?

According to the article, it would be difficult to tie any request to a user: > Target diffusion starts with the request metadata, which leaves out any personally identifiable information about the source device or user, and includes only limited contextual data about the request that’s required to enable routing to the appropriate model If this is the case, I wonder how the authentication would work. Is it a securit…

Don't know if they use it (or if it would somehow weaken/break the privacy claims you cited), but Apple has an SDK called DeviceCheck[0].

Essentially, your server send a nonce which the client signs using a key pair derived from the Secure Enclave. The server can then verify the signature by an API provided by Apple's servers, and they respond whether it was signed by a Secure Enclave resident key or not.

I'm guessing this could be helpful to make it hard(er) to write a Python client.

[0]: https://developer.apple.com/documentation/devicecheck/establ...

Re: Private Cloud Compute: A new frontier for AI privacy in the cloud

#254

Post by KaiserPro »

amiantos wrote: post #119
ein0p wrote: post #78

It is not possible for this to be fully private in the United States because the government not only can force Apple to open up the kimono, it can also forbid it to talk about it. There’s not really anything Apple can do to work around this “limitation”. Thank your “representative” for extending the PATRIOT Act when you get a chance.

Private Cloud Compute servers have no persistent storage so there would be nothing to see upon opening the kimono. You'd need some sort of government requested live wire tap thing to harvest the data out of the incoming requests, which might be a different situation. I'm, of course, just some dude on the internet, thinking up a counter-point to this concern, who knows if I am even remotely in the right ballpark.

> Private Cloud Compute servers have no persistent storage so there would be nothing to see upon opening the kimono

It doesn't actually say there is no persistent storage, it says that the compute node will not store it for longer than the request. There's nothing to stop the data coming from a datastore outside of the "PCC" in another part of apple's infrastructure.

Re: Private Cloud Compute: A new frontier for AI privacy in the cloud

#255

Post by ADeerAppeared »

nl wrote: post #251
hexage1814 wrote: post #168

The thing with cloud and with anything related to it, anything that connects to the internet somehow... is that, unless it's open source and the servers decentralized, you are always trusting SOMEONE. Sure, Apple might make their best to ensure nobody – but them – have access to your data... but Apple controls all the end points. It controls the updates your iPhone receives, it controls the servers where this happens…

This isn't right. If you trust math you can prove the software is what they say it is. Yes it is work to do this, but this is a big step forward.

The only thing the math tells you is that the server software gave you a correct key.

It does not tell you how it got that key. A compromised server would send you the key all the same.

You still have to trust in the security infrastructure. Trust that Apple is running the hardware it says it is, Trust that apple is running the software it says it is.

Security audits help build that trust, but it is not and never will be proof. A three-letter-agency of choice can still walk in and demand they change things without telling anyone. (And while that particular risk is irrelevant to most users, various countries are still opposed to the US having that power over such critical user data.)

Re: Private Cloud Compute: A new frontier for AI privacy in the cloud

#256

Post by tzs »

> The Secure Enclave randomizes the data volume’s encryption keys on every reboot and does not persist these random keys, ensuring that data written to the data volume cannot be retained across reboot. In other words, there is an enforceable guarantee that the data volume is cryptographically erased every time the PCC node’s Secure Enclave Processor reboots.

I wonder if there is anything that enforces an upper limit on the time between reboots?

Since they are building their own chips it would be interesting to include a watchdog timer that runs off an internal oscillator, cannot be disabled by software, and forces a reboot when it expires.

Re: Private Cloud Compute: A new frontier for AI privacy in the cloud

#257

Post by Spooky23 »

chem83 wrote: post #189
noahtallen wrote: post #178

Earlier quoted context omitted.

I don’t think that’s completely fair. It basically puts Apple in the same bucket as Google or OpenAI. Google obviously tracks everything you do for ads, recommendations, AI, you name it. They don’t even hide it, it’s a core part of their business model. Apple, on the other hand, has made a pretty serious effort to ensure that no employee can access your data on these AI systems. That’s hugely different! They’re going…

What makes you think that internal access control at Apple is any better than Google's, Microsoft's or OpenAI's? Google employees have long reported that you can't access user data with standard credentials, for example. Also, what makes you think that Apple's investments on chip design and OS is superior to Google's? Google is known for OpenTitan and other in-house silicon projects. It's also been working in secure…

It’s not about technology. It’s about their business.

Apple generally engineers their business so that there isn’t an incentive to violate those access controls or principles. Thats not where the money is for them.

Behavior is always shaped by rewards and punishments. Positive reinforcement is always stronger.

Re: Private Cloud Compute: A new frontier for AI privacy in the cloud

#258

Post by yolovoe »

WatchDog wrote: post #147

I'm interested in how this compares to AWS nitro enclaves, which they mention briefly. The main difference seems to be verifiability down to the firmware level. Nitro enclaves does not provide measurements of the firmware[0], or hypervisor, furthermore they state that the hypervisor code can be updated transparently at any time[1]. Apple is going to provide images of the secure enclave processor operating system(sepO…

Nitro does measure firmware. If any firmware is unexpected, server will essentially stop being connected to the EC2 substrate network and/or server wiped clean automatically. People will be paged automatically, security will likely be pulled in, etc.

There is no reason to measure hypervisor firmware as it’s not firmware in the case of EC2. The BIOS/UEFI firmware on the mobo is overwritten if it’s tampered with. Hypervisor code (always signed, like all code) is streamed via a verifiably secure system on the server (Nitro cards, which make use of measured boot and/or secure boot).

No idea what the customer facing term “Nitro enclaves” means, but EC2 engineers are literally mobilized like an army with pages when any security risk (even minor ones) is determined. Basic stuff like this is covered. We even go as far as guaranteeing core dumps don’t contain any real customer data, even encrypted

Re: Private Cloud Compute: A new frontier for AI privacy in the cloud

#259

Post by manmal »

al_borland wrote: post #137
gpm wrote: post #125

Earlier quoted context omitted.

I actually thought one notable thing in the presentation was that they spent all this time talking about their new private cloud compute architecture. And then showed that they have a prompt asking if you're ok sending the data to OpenAI. Presumably because despite OpenAI promising not to use your data (a promise apple relayed) OpenAI didn't buy into this new architecture.

Thank you for mentioning this. I thought I was going crazy, because I heard this too, but kept seeing comment after comment on other sites asking if a person could choose not to use OpenAI, or that it was happening magically in the background. The way I heard it, the user was in control. I think this goes back to what Steve said in 2010. https://youtube.com/watch?v=Ij-jlF98SzA And yes, while the data might not be lin…

I’m not sure but I thought I saw it mentioned that OpenAI is still allowed to train on the data received from Apple customers.

Re: Private Cloud Compute: A new frontier for AI privacy in the cloud

#260

Post by gpm »

jahewson wrote: post #175
gpm wrote: post #152

Earlier quoted context omitted.

> In this option it would be Apple releasing a malicious software update? Yes, compelled by something like the all writs act (if the US is the one doing the compelling). > This option reads like the keys are stored in apple-keys.txt They probably are. That file might live on a CD drive in a safe that requires two people to open it, but ultimately it's a short chunk of binary data that exists somewhere (until it is de…

Allow me to introduce you to HSMs: https://en.m.wikipedia.org/wiki/Hardware_security_module

I'm well aware of these. They don't solve the problem at hand. You need a way to put keys into new hardware. Thus you need a way to get keys out of wherever you've stored your cryptographic material. Thus it can't be on a HSM (or it can be if it's a master key signing child keys, but in that case the attack only needs a signed child key).
Post reply on HN