EFF Joins Global Coalition Asking Apple CEO Tim Cook to Stop Phone-Scanning • Hacker News Board

Re: EFF Joins Global Coalition Asking Apple CEO Tim Cook to Stop Phone-Scanning

#51

Post by slownews45 » Sat, Aug 21, 2021, 8:25 PM UTC

zepto wrote: post #19

slownews45 wrote: post #16

Earlier quoted context omitted.

All good points. Yeah, we should stop using the term I think given it's lost a lot of its meaning. But a fair bit of the discussion of this topic has been badly misinformed.

I agree, but that is a feature of this particular problem. For example, I see a lot of people who are simply wrong about how the hash matching works. One way to be wrong is to think it’s a cryptographic hash. Another way to be wrong is to think it’s just a perceptual hash match. The problem is that these are both not crazy. The actual solution is far more complex and non-obvious than most people would suspect. I thin…

Apple have been pretty clear that a human will review things. That is (I hope) a feature of any system that leads to child porn charges. If not it should be - AI gets things wrong and can be tricked (as can humans but in different ways usually).

But agreed, the technical bits may not be obvious (though apple released I thought a pretty darn complete paper on how it all works).

Re: EFF Joins Global Coalition Asking Apple CEO Tim Cook to Stop Phone-Scanning

#52

Post by sharken » Sat, Aug 21, 2021, 8:25 PM UTC

hoppyhoppy2 wrote: post #34

bequanna wrote: post #15

Does Tim Cook have a choice here? I would be surprised to hear that the genesis of this idea was inside of Apple vs. one or more govts pressuring Apple to add this functionality for them. It is also likely they even suggested that Apple should market this as anti-pedo tech to receive the least pushback from users.

The latest episode of The Daily podcast [0] from The New York Times said that Apple executives were told by members of Congress at a hearing that if they didn't do something about CSAM on their platform the federal government would force them through legislation. And it's not a completely idle threat; just look at the proposed EARN-IT Act of 2020 [1], which would pretty much outlaw end-to-end encrypted services witho…

Be that as it may, there has not been presented any proof that Congress would dictate client-side scanning on users devices.

But it still doesn't change that Apple needs to stop on-device scanning.

Re: EFF Joins Global Coalition Asking Apple CEO Tim Cook to Stop Phone-Scanning

#53

Post by azinman2 » Sat, Aug 21, 2021, 8:28 PM UTC

nullc wrote: post #45

azinman2 wrote: post #35

Earlier quoted context omitted.

You’d have to not only have over 30 hash collisions, but also have it collide with another secret hash function, and then also have a human look at it and agree it’s CP. So what’s the actual realistic issue here? This keeps getting thrown around as if it’s likely, yet not only are there numerous steps against this in the Apple chain, this would already be a huge issue with Dropbox, Facebook, Microsoft, Google, etc wh…

> You’d have to not only have over 30 hash collisions That's trivial. If the attacker can get one image onto your device they can get several. It's very easy to construct preimages for Apple's neural hash function, including fairly good looking ones (e.g. https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue... ) > collide with another secret hash function The supposed other 'secret' hash function cannot be se…

If this really was such a problem, then as I said, we’d have been getting reports of this over the past 10+ years it’s already been in place at big cloud providers. So where is all of this ruining of peoples lives by uploading CP on their devices?

Also if you’re a gov actor trying to frame someone, why bother with a pre-image when you could put the real images on it?

None of that is new today — all that’s new is Apple is joining the effort to scan for CSAM, and instead of doing it on server they’re doing it on device right before you upload in a way that attempts to be more secure and private than other efforts.

Re: EFF Joins Global Coalition Asking Apple CEO Tim Cook to Stop Phone-Scanning

#54

Post by pfranz » Sat, Aug 21, 2021, 8:28 PM UTC

carlosdp wrote: post #14

azinman2 wrote: post #11

> As we’ve explained in Deeplinks blog posts, Apple’s planned phone-scanning system opens the door to broader abuses. It decreases privacy for all iCloud photo users, and the parental notification system is a shift away from strong end-to-end encryption. It will tempt liberal democratic regimes to increase surveillance, and likely bring even great pressures from regimes that already have online censorship ensconced i…

> (though that could easily be the same even without all these CSAM measures.. not a new threat) Apple has historically avoided being pressured by governments to allow this kind of surveillance by arguing they can't be forced to create functionality that doesn't exist, or hand over information they don't have. It's the argument they made in the San Bernadino case. If they release this, it'll be much harder to avoid g…

Ehh. I'd argue Apple has a mixed record. It's well known that iCloud backups are unencrypted and often handed over to authorities. In Jan 2020[1], it was reported they planned to encrypt backups, but dropped the rollout due to pressure from the FBI. I'm surprised they don't get called out because the difference between this and San Bernadino makes sense to me from a technical standpoint, from a practical standpoint and to the laymen it seems hypocritical.

In this case, I actually kind of buy Apple's argument that this will make it harder to cowtow to governments (assuming they encrypt iCloud photos at some point). Right now they can scan iCloud data and hand over photos and accounts without user's knowledge. They can do that without informing users (like they currently do with backups). With this in place the database and logic ships with the OS. They would have to implement and ship any changes world-wide. Users will have to install that update. An alternative is Apple silently making a server-side change affecting specific customers or countries. With that said, I do understand people's concern over the change.

[1] https://www.reuters.com/article/us-apple-fbi-icloud-exclusiv...

Re: EFF Joins Global Coalition Asking Apple CEO Tim Cook to Stop Phone-Scanning

#55

Post by tzs » Sat, Aug 21, 2021, 8:30 PM UTC

nullc wrote: post #45

azinman2 wrote: post #35

Earlier quoted context omitted.

You’d have to not only have over 30 hash collisions, but also have it collide with another secret hash function, and then also have a human look at it and agree it’s CP. So what’s the actual realistic issue here? This keeps getting thrown around as if it’s likely, yet not only are there numerous steps against this in the Apple chain, this would already be a huge issue with Dropbox, Facebook, Microsoft, Google, etc wh…

> You’d have to not only have over 30 hash collisions That's trivial. If the attacker can get one image onto your device they can get several. It's very easy to construct preimages for Apple's neural hash function, including fairly good looking ones (e.g. https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue... ) > collide with another secret hash function The supposed other 'secret' hash function cannot be se…

> That's trivial. If the attacker can get one image onto your device they can get several.

At which point everything you brought up about attacks on the hash function is completely irrelevant because the attacker can put actual child porn from the database on your device.

Re: EFF Joins Global Coalition Asking Apple CEO Tim Cook to Stop Phone-Scanning

#56

Post by feanaro » Sat, Aug 21, 2021, 8:34 PM UTC

azinman2 wrote: post #44

sennight wrote: post #37

Earlier quoted context omitted.

> What more could one ask for? An independent audit for both the secret secondary perceptual hashing algorithm and the chain of custody policies/compliance for the "US db" and the disconcertedly open ended "not yet chosen non-US db"?

Do you ask for the same audit at Facebook, Google, Microsoft, Dropbox, and countless others who are already doing this and have been for years? I do not share your same concern of some abused db _today_.

We should definitely start. Also, do those companies implement such subversive technology in devices they sell you?

Re: EFF Joins Global Coalition Asking Apple CEO Tim Cook to Stop Phone-Scanning

#57

Post by azinman2 » Sat, Aug 21, 2021, 8:34 PM UTC

amadeuspagel wrote: post #47

azinman2 wrote: post #44

Earlier quoted context omitted.

Do you ask for the same audit at Facebook, Google, Microsoft, Dropbox, and countless others who are already doing this and have been for years? I do not share your same concern of some abused db _today_.

Neither Google nor Microsoft scan pictures people have on their devices running Android or Windows. I'm not sure how that's even applicable to Facebook and Dropbox.

You’re asking for an auditing chain presumably due to concerns about governments putting in photos of things that aren’t CP. Apple is only doing this for photos that get uploaded to iCloud, with this neural hash that gets uploaded with it. The actual verification of a CP match occurs on the server due to the hashes being blinded. So in many ways, it’s very similar to what these other cloud providers effectively do — search for CP matches on uploaded data.

If you’re concerned about non-CP being scanned for, then you should already be concerned about that with everyone else. Thus if you’re asking for auditing of Apple, then you should widen your request. If you do, then sure, I can understand that. If you’re not concerned about the existing system, then I think you’re being non-consistent.

Most people in comments to me seem to be non-consistent, and are being overly knee jerk about this… myself included initially.

Re: EFF Joins Global Coalition Asking Apple CEO Tim Cook to Stop Phone-Scanning

#58

Post by ajsnigrutin » Sat, Aug 21, 2021, 8:34 PM UTC

shuckles wrote: post #30

carlosdp wrote: post #14

Earlier quoted context omitted.

> (though that could easily be the same even without all these CSAM measures.. not a new threat) Apple has historically avoided being pressured by governments to allow this kind of surveillance by arguing they can't be forced to create functionality that doesn't exist, or hand over information they don't have. It's the argument they made in the San Bernadino case. If they release this, it'll be much harder to avoid g…

The request from the FBI in the San Bernardino case was to change a passcode limit constant and retry timeouts. Those are about as trivial to implement as any of the convoluted government coercion database attacks against CSAM detection being proposed here.

The difference here is, that apple would have to develop a new feature for them, test it, and waste millions in lawers to protect themselves from accusations of tampering with the evidence (which a software update definitely is, and who knows what FBI wanted in that software update, maybe even to insert a fake sms to the sms database, or many other things a good defense lawyer could bring to the jury).

Here, it's different.. let's say there's a new wikileaks, photos of secret documents... and again, first a few journalists get the data, start slowly writing articles, and FBI just adds the hashes to the database, and they can find out who has the photos, with metadata even who had the first, before the journalist, and they can find a leak.

Re: EFF Joins Global Coalition Asking Apple CEO Tim Cook to Stop Phone-Scanning

#59

Post by lamontcg » Sat, Aug 21, 2021, 8:36 PM UTC

azinman2 wrote: post #50

lamontcg wrote: post #42

Earlier quoted context omitted.

> It only weakens privacy for iCloud photos if you have CP or photos in a CP database Or if someone hacks your device and uploads one of those photos to your iCloud. (I still have no idea why people aren't pointing this out more aggressively -- phones get hacked probably every minute of every day, and acquiring those actual photos isn't that difficult once you're willing to commit felonies -- hash collisions are a di…

Because that then would already be a problem for Facebook, Google, Microsoft, etc that host photos that hacked phones could be uploading today. And we’re just not seeing that being the case. Because all these providers have been doing this for so many years, including the nearly 17 million photos identified by Facebook last year, you’d figure there would be a lot more noise if this was really going on. In fact, I wou…

We had SWAT teams for a long time before SWATing became popular. The publicity that this has gotten is only going to increase the chances that all these services start getting abused. And who is to say that it hasn't happened already and been entirely successful, but nobody believed the victim.

Re: EFF Joins Global Coalition Asking Apple CEO Tim Cook to Stop Phone-Scanning

#60

Post by sennight » Sat, Aug 21, 2021, 8:36 PM UTC

azinman2 wrote: post #44

sennight wrote: post #37

Earlier quoted context omitted.

> What more could one ask for? An independent audit for both the secret secondary perceptual hashing algorithm and the chain of custody policies/compliance for the "US db" and the disconcertedly open ended "not yet chosen non-US db"?

Do you ask for the same audit at Facebook, Google, Microsoft, Dropbox, and countless others who are already doing this and have been for years? I do not share your same concern of some abused db _today_.

Sure, but I don't expect it from third party cloud platforms - in the same way I wouldn't expect accountability from a garbage man who reports to the police after finding evidence of crime in my garbage. Apple is, for some insane reason, trying to establish the precedent that the contents of your Apple product are now part of the public space - where expectation of privacy isn't a thing.