Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Security research on Private Cloud Compute

security.apple.com

Post reply on HN
Print view View original

71–80 of 124 posts

Re: Security research on Private Cloud Compute

#71

Post by gigel82 »

davidczech wrote: post #70
gigel82 wrote: post #67

Earlier quoted context omitted.

Ok, but they write and fully control the closed-source software that appends to the log. How can anyone verify that all the code paths append to the log? I'm pretty sure they can just not append to the log from their ExfiltrateDataForAdvertisment() and ExfiltrateDataForGovernments() functions. Maybe I'm not being clear; transparent logs solve the problem of supply chain attacks (that is, Apple can use the logs to som…

> How can anyone verify that all the code paths append to the log? I'm pretty sure they can just not append to the log from their ExfiltrateDataForAdvertisment() and ExfiltrateDataForGovernments() functions. I think we have different understandings of what the transparency log is utilized for. The log is used effectively as an append-only hash set of trusted software hashes a PCC node is allowed to run, accomplished…

One of the replies in this thread sent me to transparency.dev which describes transparency logs as something different. But reading Apple's description doesn't change my opinion on this. It is a supply-chain / MITM protection measure and does absolutely nothing to assuage my privacy concerns.

Bottom line, I just hope that there will be a big checkbox in the iPhone's settings that completely turns off all "cloud compute" for AI scenarios (checked by default) and I hope it gets respected everywhere. But they're making such a big deal of how "private" this data exfiltration service is that I fear they plan to just make it default on (or not even provide an opt-out at all).

Re: Security research on Private Cloud Compute

#72

Post by astrange »

gigel82 wrote: post #69
astrange wrote: post #65

Earlier quoted context omitted.

> They control the servers and services entirely There's a key signing ceremony with a third-party auditor watching; it seems to rely on trusting them together with the secure boot process. But there are other things you can add to this, basically along the lines of making the machine continually prove that it behaves like the system described in the log. They don't control all of the service though; part of the syst…

"ceremony" is a good choice of word; it's all ceremonial and nonsense; as long as they control the hardware and the software there is absolutely no way for someone to verify this claim. Apple has lied to shareholders before, remember those "what happens on your iPhone, stays on your iPhone" billboards back in the day they used to fool everyone into thinking Apple cares about privacy? A couple years later, they were p…

> Apple has lied to shareholders before

So sue them.

> how everyone's iPhone will scan their files and literally send them to law enforcement

That was a solution for if you opted into a cloud service, was a strict privacy improvement because it came alongside end-to-end encryption in the cloud, and I think was mandated by upcoming EU regulations (although I think they changed the regulations so it was dropped.)

Note in the US service providers are required to report CSAM to NCMEC if they see it; it's literally the only thing they're required to do. But NCMEC is not "law enforcement" or "government", it's a private organization specially named in the law. Very important distinction because if anyone does give your private information to law enforcement you'd lose your 4th Amendment rights over it, since the government can share it with itself.

(I think it may actually be illegal to proactively send PII to law enforcement without them getting a subpoena first, but don't remember. There's an exception for emergency situations, and those self service portals that large corporations have are definitely questionable here.)

Re: Security research on Private Cloud Compute

#73

Post by astrange »

saagarjha wrote: post #57
astrange wrote: post #55

Earlier quoted context omitted.

That's detectable through timing measurements, for the same reason you can't have data-dependent operations in cryptography.

Ok, where are the timing measurements?

In the hardware secure boot chain ;)

You do have to trust the SEP/TPM here, it sounds like. That is verified by having a third party auditor watch them get installed, and by the anonymous proxy routing thingy making it so they can't fake only some of them but would have to fake all of them to be reliable.

If they were okay with it being unreliable, then clients could tell via timing because some of the nodes would perform differently, or they'd perform differently depending on which client or what prompt it was processing. It's surprisingly difficult to hide timing differences, eg all those Spectre cache-reading attacks on browsers.

It does look like there's room to add more verification (like the client asking the server to do more intensive proofs, or homomorphic encryption). Could always go ask for it.

Re: Security research on Private Cloud Compute

#74

Post by astrange »

ls612 wrote: post #62
astrange wrote: post #56

Earlier quoted context omitted.

> There is absolutely no way for a customer to verify their claims that the data is not saved or transferred elsewhere. Transparency logs are capable of verifying that, it's more or less the whole point of them. (Strictly speaking, you can make it arbitrarily expensive to fake it.) Also, if they were "transferring your data elsewhere" it would be a GDPR violation. Ironically wrt your China claim, it would also be ill…

Are transparency logs akin to Certificate Transparency but for signed code? I’ve read through the section a couple times and still don’t fully understand it.

Yeah, it's a log of all the software that runs on the server. If you trust the secure boot process then you trust the log describes its contents.

If you don't trust the boot process/code signing system then you'd want to do something else, like ask the server to show you parts of its memory on demand in case you catch it lying to you. (Not sure if that's doable here because the server has other people's data on it, which is the whole point.)

Re: Security research on Private Cloud Compute

#75

Post by miki123211 »

dewey wrote: post #2

Looks like they are really writing everything in Swift on the server side. Repo: https://github.com/apple/security-pcc

It's worth keeping in mind that these AI machines run an environment very similar to Mac OS, XNU kernel and all, and are powered by Apple Silicon. Using Swift in that context makes sense.

At least according to what we publicly know, no other backend Apple services follow this model.

Re: Security research on Private Cloud Compute

#76

Post by gigel82 »

stouset wrote: post #45
mmastrac wrote: post #5

I feel like this is all smoke and mirrors to redirect from the likelihood intentional silicon backdoors that are effectively undetectable. Without open silicon, there's no way to detect that -- say -- when registers r0-rN are set to values [A, ..., N] and a jump to address 0xCONSTANT occurs, additional access is granted to a monitor process. Of course, this limits the potential attackers to 1) exactly one government…

If you take as a fundamental assumption that all your hardware is backdoored by Mossad who has unlimited resources and capacity to intercept and process all your traffic, the game is already lost and there’s no point in doing anything. If instead you assume your attackers have limited resources, things like this increase the costs attackers have to spend to compromise targets, reducing the number of viable targets an…

Some of us just assume Apple itself is a bad actor planning to use and sell customer data for profit; makes all of this smoke and mirrors like GP said.

There is absolutely no technical solution where Apple can prove our data isn't exfiltrated as long as this is their software that runs on their hardware.

Re: Security research on Private Cloud Compute

#77

Post by davidczech »

gigel82 wrote: post #71
davidczech wrote: post #70

Earlier quoted context omitted.

> How can anyone verify that all the code paths append to the log? I'm pretty sure they can just not append to the log from their ExfiltrateDataForAdvertisment() and ExfiltrateDataForGovernments() functions. I think we have different understandings of what the transparency log is utilized for. The log is used effectively as an append-only hash set of trusted software hashes a PCC node is allowed to run, accomplished…

One of the replies in this thread sent me to transparency.dev which describes transparency logs as something different. But reading Apple's description doesn't change my opinion on this. It is a supply-chain / MITM protection measure and does absolutely nothing to assuage my privacy concerns. Bottom line, I just hope that there will be a big checkbox in the iPhone's settings that completely turns off all "cloud compu…

> It is a supply-chain / MITM protection measure

It is so much more than that, but you are entitled to your own opinion.

Re: Security research on Private Cloud Compute

#78

Post by willtemperley »

danielhep wrote: post #22
dewey wrote: post #2

Looks like they are really writing everything in Swift on the server side. Repo: https://github.com/apple/security-pcc

Is using something other than XCode viable? I'd love to do more with swift but I hate that IDE.

Have you used it recently, on an M series Mac? I used to feel the same, it was sluggish and crashed frequently. It's become usable now, even pleasant to use. Also it's great they support Vim keybindings now out-of-the-box.

Re: Security research on Private Cloud Compute

#79

Post by Jerrrrrrry »

stouset wrote: post #45
mmastrac wrote: post #5

I feel like this is all smoke and mirrors to redirect from the likelihood intentional silicon backdoors that are effectively undetectable. Without open silicon, there's no way to detect that -- say -- when registers r0-rN are set to values [A, ..., N] and a jump to address 0xCONSTANT occurs, additional access is granted to a monitor process. Of course, this limits the potential attackers to 1) exactly one government…

If you take as a fundamental assumption that all your hardware is backdoored by Mossad who has unlimited resources and capacity to intercept and process all your traffic, the game is already lost and there’s no point in doing anything. If instead you assume your attackers have limited resources, things like this increase the costs attackers have to spend to compromise targets, reducing the number of viable targets an…

Soviets used typewriters.

American Lawyers of the highest pedigree (HNWI) don't even use email.

Your hardware is back-doored, as Intel is named "Intel" for a (nearly too poignant) reason.

Re: Security research on Private Cloud Compute

#80

Post by Jerrrrrrry »

kmeisthax wrote: post #36
mmastrac wrote: post #5

I feel like this is all smoke and mirrors to redirect from the likelihood intentional silicon backdoors that are effectively undetectable. Without open silicon, there's no way to detect that -- say -- when registers r0-rN are set to values [A, ..., N] and a jump to address 0xCONSTANT occurs, additional access is granted to a monitor process. Of course, this limits the potential attackers to 1) exactly one government…

The economics of silicon manufacturing and Apple's own security goals (including the security of their business model) restrict the kinds of backdoors you can embed in their servers at that level. Let's assume Apple has been compromised in some way and releases new chips with a backdoor. It's expensive to insert extra logic into just one particular spin of a chip; that involves extra tooling cost that would be notice…

  >backdoors inevitably leak? Well, now you have a trivial jailbreak vector
the discover-ability of an exploit vector relates little to its trivialness, definitely when considering the context (nation-state-APTs)

You can hold the enter key down for 40 seconds to login into any certain Linux Server distro, for years. No one knew, ez to do.

You can have a chip inside your chip that only accepts encrypted and signed microcode and has control over the superior chip. Everyone knows - nothing you can do.

Nation state actors however, can facilitate either; APT's can forge fake digital forensics that imply another motive/state/false flag.

Post reply on HN