Did Apple say anything about what training data they used for their generative image models?
Yes, basically if you opted out of Apple scraping, your data isn’t used
Private Cloud Compute: A new frontier for AI privacy in the cloud
11–20 of 393 posts
Re: Private Cloud Compute: A new frontier for AI privacy in the cloud
#12This entire platform is the first time I've strategically considered realigning the majority of my use to Apple. Airtag anonymity was pretty cool, technically speaking, but a peripheral use case for me. To me, PCC is a well-reasoned, surprisingly customer-centric response to the fact that due to (processing, storage, battery) limitations not all useful models can be run on-device. And they tried to build a privacy ar…
It's very encouraging.
Another good step in this direction would be publishing a list of all on-device Apple software (including Spotlight models for image analysis) and details of any information that is sent to Apple, along with opt-out instructions via device Settings or Apple Configurator MDM profiles.
Apple does publish a list of network ports and servers, so that network traffic can be permitted for specific services. The list is complicated by 3rd-party CDNs, but can be made to work with dnsmasq and ipset, "Use Apple products on enterprise networks", https://support.apple.com/en-us/101555
Re: Private Cloud Compute: A new frontier for AI privacy in the cloud
#13> In a first for any Apple platform, PCC images will include the sepOS firmware and the iBoot bootloader in plaintext, making it easier than ever for researchers to study these critical components.
Yes!
> Software will be published within 90 days of inclusion in the log, or after relevant software updates are available, whichever is sooner.
I think this theoretically leaves a 90-day maximum gap between publishing vulnerable software and potential-for-discovery. I sincerely hope that the actual availability of images is closer to instant than the maximum, though.
Re: Private Cloud Compute: A new frontier for AI privacy in the cloud
#14Earlier quoted context omitted.
Do you have analogous search terms for Microsoft, Alphabet, Google, and Amazon's approaches? Your comment makes me curious on how guarantee-to-guarantee looks (and associated architectures).
https://cloud.google.com/docs/security/binary-authorization-... is one example
Apple's system goes further by having incoming requests choose and verify a server and then encrypt itself using the public key of the node to prevent MITM attacks.
And a one-time credential to prevent replay attacks.
As well as minor things like obfuscating IP addresses, metadata etc.
Re: Private Cloud Compute: A new frontier for AI privacy in the cloud
#15Sounds good. Still won’t send anything sensitive there but I appreciate the effort and direction, especially when current industry trend seems to be fuck you were rewriting our TOS to take your data.
Re: Private Cloud Compute: A new frontier for AI privacy in the cloud
#16I really want to see this OS, and have cautious optimism that this could be the first time we'll see a big tech company actually provide an auditable security guarantee!
I think depending on how this plays out, Apple might manage to earn some of the trust its users have in it, which would be pretty cool! But even cooler will be if we get full chain-of-custody audits, which I think will have to entail opening up some other bits of their stack
In particular, the cloud OS being open-source, if they make good on that commitment, will be incredibly valuable. My main concern right now is that if virtualization is employed in their actual deployment, there could be a backdoor that passes keys from secure enclaves in still-proprietary parts of the OSes running on user devices to a hypervisor we didn't audit that can access the containers. Surely people with more security expertise than me will have even better questions.
Maybe Apple will be responsive to feedback from researchers and this could lead to more of this toolchain being auditable. But even if we can't verify that their sanctioned use case is secure, the cloud OS could be a great step forward in secure inference and secure clouds, which people could independently host or build an independent derivative of
The worst case is still that they just don't actually do it, but it seems reasonably likely they'll follow through on at least that, and then the worst case becomes "Super informative open-source codebase for secure computing at scale just dropped" which is a great thing no matter how the other stuff goes
Re: Private Cloud Compute: A new frontier for AI privacy in the cloud
#17Earlier quoted context omitted.
https://cloud.google.com/docs/security/binary-authorization-... is one example
Apple's system goes further by having incoming requests choose and verify a server and then encrypt itself using the public key of the node to prevent MITM attacks. And a one-time credential to prevent replay attacks. As well as minor things like obfuscating IP addresses, metadata etc.
Apples system is also the entire pipeline. Borg SREs can still change behavior here. It’s a lot better than what most places have but does not go far enough.
Re: Private Cloud Compute: A new frontier for AI privacy in the cloud
#18I really want to see this OS, and have cautious optimism that this could be the first time we'll see a big tech company actually provide an auditable security guarantee! I think depending on how this plays out, Apple might manage to earn some of the trust its users have in it, which would be pretty cool! But even cooler will be if we get full chain-of-custody audits, which I think will have to entail opening up some…
> even if we can't verify that their sanctioned use case is secure, the cloud OS could be a great step forward in secure inference and secure clouds, which people could independently host or build an independent derivative of
Yes, the tech industry loves to copy Apple :)
Asahi Linux has a good overview of on-device boot chain security, https://github.com/AsahiLinux/docs/wiki/Apple-Platform-Secur...
> My main concern right now is that if virtualization is employed in their actual deployment, there could be a backdoor that passes keys from secure enclaves in still-proprietary parts of the OSes running on user devices to a hypervisor we didn't audit that can access the containers.
We’ll release a PCC Virtual Research Environment: a set of tools and images that simulate a PCC node on a Mac with Apple silicon, and that can boot a version of PCC software minimally modified for successful virtualization.
Could a PCC node be simulated on iPad Pro with M4 Apple Silicon?
Re: Private Cloud Compute: A new frontier for AI privacy in the cloud
#19Earlier quoted context omitted.
Do you have analogous search terms for Microsoft, Alphabet, Google, and Amazon's approaches? Your comment makes me curious on how guarantee-to-guarantee looks (and associated architectures).
Most of the stuff in the blog post reads like common security precautions: don't run as root, stateless immutable nodes, use secure boot, etc. All wrapped up in some Apple marketing pizzazz.
> common security precautions ... marketing pizzazz.
If it were this common, Meta, Google, and others would have announced or launched something similar for its consumer apps/services; I can't seem to recall anything of note.
Re: Private Cloud Compute: A new frontier for AI privacy in the cloud
#20I really want to see this OS, and have cautious optimism that this could be the first time we'll see a big tech company actually provide an auditable security guarantee! I think depending on how this plays out, Apple might manage to earn some of the trust its users have in it, which would be pretty cool! But even cooler will be if we get full chain-of-custody audits, which I think will have to entail opening up some…
> could be the first time we'll see a big tech company actually provide an auditable security guarantee
AWS Nitro Enclaves [0] come close but of course what Apple has done is productize private compute for its 1b+ macOS & iOS customers!
[0] https://docs.aws.amazon.com/enclaves/latest/user/nitro-encla...