261–270 of 393 posts
Earlier quoted context omitted.
Slightly off-topic, "open up the kimono" sounds disturbing and creepy to me as an Asian. I suspect I'm not alone in this.
Some share your sentiment. https://www.npr.org/sections/codeswitch/2014/11/02/360479744...
Earlier quoted context omitted.
This isn't right. If you trust math you can prove the software is what they say it is. Yes it is work to do this, but this is a big step forward.
The only thing the math tells you is that the server software gave you a correct key. It does not tell you how it got that key. A compromised server would send you the key all the same. You still have to trust in the security infrastructure. Trust that Apple is running the hardware it says it is, Trust that apple is running the software it says it is. Security audits help build that trust, but it is not and never wil…
To quote:
verifiable transparency, goes one step further and does away with the hypothetical: security researchers must be able to verify the security and privacy guarantees of Private Cloud Compute, and they must be able to verify that the software that’s running in the PCC production environment is the same as the software they inspected when verifying the guarantees.
So how does this work?
> The PCC client on the user’s device then encrypts this request directly to the public keys of the PCC nodes that it has first confirmed are valid and cryptographically certified. This provides end-to-end encryption from the user’s device to the validated PCC nodes, ensuring the request cannot be accessed in transit by anything outside those highly protected PCC nodes
> Next, we must protect the integrity of the PCC node and prevent any tampering with the keys used by PCC to decrypt user requests. The system uses Secure Boot and Code Signing for an enforceable guarantee that only authorized and cryptographically measured code is executable on the node. All code that can run on the node must be part of a trust cache that has been signed by Apple, approved for that specific PCC node, and loaded by the Secure Enclave such that it cannot be changed or amended at runtime.
But why can't a 3-letter agency bypass this?
> We designed Private Cloud Compute to ensure that privileged access doesn’t allow anyone to bypass our stateless computation guarantees.
> We consider allowing security researchers to verify the end-to-end security and privacy guarantees of Private Cloud Compute to be a critical requirement for ongoing public trust in the system.... When we launch Private Cloud Compute, we’ll take the extraordinary step of making software images of every production build of PCC publicly available for security research. This promise, too, is an enforceable guarantee: user devices will be willing to send data only to PCC nodes that can cryptographically attest to running publicly listed software.
So your data will not be sent to node that are not cryptographically attested by third parties.
These are pretty strong guarantees, and really make it difficult for Apple to bypass.
It's like end-to-end encryption using the Signal protocol: relatively easy to verify it is doing what is claimed, and extraordinarily hard to bypass.
Specifically:
> The only thing the math tells you is that the server software gave you a correct key.
No, this is secure attestation. See for example https://courses.cs.washington.edu/courses/csep590/06wi/final... which explains it quite well.
The weakness of attestation is that you don't know what the root of trust is. But Apple strengthens this by their public inspection and public transparency logs, as well as the target diffusion technique which forces an attack to be very widespread to target a single user.
These aren't simple things for a 3LA to work around.
Earlier quoted context omitted.
What makes you think that internal access control at Apple is any better than Google's, Microsoft's or OpenAI's? Google employees have long reported that you can't access user data with standard credentials, for example. Also, what makes you think that Apple's investments on chip design and OS is superior to Google's? Google is known for OpenTitan and other in-house silicon projects. It's also been working in secure…
It’s not about technology. It’s about their business. Apple generally engineers their business so that there isn’t an incentive to violate those access controls or principles. Thats not where the money is for them. Behavior is always shaped by rewards and punishments. Positive reinforcement is always stronger.
All these conversations always end up boiling down to someone thinking they’re being clever for pointing out you have to trust a company at the end of the day when it comes to security and privacy.
Yes. Valid. So if you have to trust someone, doesn’t it make sense for it to be someone who has built protecting privacy into their core value proposition, versus a company that has baked violating your privacy into their value prop?
Earlier quoted context omitted.
> unless it's open source and the servers decentralized, you are always trusting SOMEONE Specifically, open-source and self-hostable . Open source doesn't save you if people can't run their own servers, because you never know whether what's in the public repo is the exact same thing that's running on the cloud servers.
> exact same thing that's running on the cloud servers What runs on the servers isn't actually very important. Why? Becuase even if you could somehow know with 100% certainty that what a server runs is the same code you can see, any provider is still subject to all kinds of court orders. What matters is the client code. If you can audit the client code (or better yet, build your own compatible client based on API spe…
Earlier quoted context omitted.
I don’t think that’s completely fair. It basically puts Apple in the same bucket as Google or OpenAI. Google obviously tracks everything you do for ads, recommendations, AI, you name it. They don’t even hide it, it’s a core part of their business model. Apple, on the other hand, has made a pretty serious effort to ensure that no employee can access your data on these AI systems. That’s hugely different! They’re going…
"ensuring that clients will refuse to talk to non-audited systems." I'm trying to understand if this is really possible. I know they claim so but is there any info on how this would prevent Apple from executing different code to what is presented for audit?
So the question is: could the hash be falsified? That’s why they’re publishing the source code to firmware and bootloader, so researchers can audit the secure boot foundations.
I am sure there is some way that a completely malevolent Apple could design a weakness into this system so they could spend a fortune on the trappings while still being able to access user information they could never use without exposing the lie and being crushed under class actions and regulatory assault.
But I reject the idea that that remote possibility means the whole system offers no benefit users should consider in purchasing decisions.
Earlier quoted context omitted.
I don’t think that’s completely fair. It basically puts Apple in the same bucket as Google or OpenAI. Google obviously tracks everything you do for ads, recommendations, AI, you name it. They don’t even hide it, it’s a core part of their business model. Apple, on the other hand, has made a pretty serious effort to ensure that no employee can access your data on these AI systems. That’s hugely different! They’re going…
It's completely fair, because regardless of third party audits, chips, etc, there are backdoors right along the line, that are going to provide Apple and the government with secret legal access to your data. They can simply go to a secret court, receive a secret judgment, and be authorised to secretly view your data. Does anyone really think this is not already the case? There is no transparency. A licensed third par…
Under the system described in the linked paper, your scenario is not possible. In fact, the whole thing looks to be designed to prevent exactly that scenario.
Where do you see the weakness? How could a secret order result in undetectable data capture?
Earlier quoted context omitted.
I don’t think that’s completely fair. It basically puts Apple in the same bucket as Google or OpenAI. Google obviously tracks everything you do for ads, recommendations, AI, you name it. They don’t even hide it, it’s a core part of their business model. Apple, on the other hand, has made a pretty serious effort to ensure that no employee can access your data on these AI systems. That’s hugely different! They’re going…
What makes you think that internal access control at Apple is any better than Google's, Microsoft's or OpenAI's? Google employees have long reported that you can't access user data with standard credentials, for example. Also, what makes you think that Apple's investments on chip design and OS is superior to Google's? Google is known for OpenTitan and other in-house silicon projects. It's also been working in secure…
Some good comments on this from cryptographer Matt Green here: https://x.com/matthew_d_green/status/1800291897245835616?t=C... (I wonder if Matt realizes nobody can read his tweets without a X account? Use BlueSky or Masto man) Edit: here's his thread combined https://threadreaderapp.com/thread/1800291897245835616.html?...
Earlier quoted context omitted.
You can by having an attestation of the signed software components up from the secure boot process, and having the client device validate said attestation corresponds to the known public version of each component, and randomize client connections across infrastructure. Other than obvious "open source software isn't perfectly secure" attack scenarios, this would require a non-targeted hardware attack, where the entire…
Usually this is done the other way around - servers verifying client devices using a chip the manufacturer put in them and fully trusts. They can trust it, because it's virtually impossible for you (the user) to modify the behavior of this chip. However, you can't put something in Apple's server. So if you don't trust Apple, this improves the trust by... 0%. Their device says it's been attested. Has it? Who knows? Th…
> It’d be trivial to just use a fake hash
You have to go deeper to support this. Apple is publishing source code to firmware and bootloader, and the software above that is available to researchers.
The volume hash is computed way up in the stack, subject to the chain of trust from these components.
Are you suggesting that Apple will actually use totally different firmware and bootloaders, just to be able to run different system images that report fake hashes, and do so perfectly so differences between actual execution environment and attested environment cannot be detected, all while none of the executives, architects, developers, or operators involved in the sham ever leaks? And the nefarious use of the data is never noticed?
At some point this crosses over into “maybe I’m just a software simulation and the entire world and everyone in it are just constructs” territory.