Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple’s device surveillance plan is a threat to user privacy – and press freedom

freedom.press

Post reply on HN
Print view View original

71–80 of 145 posts

Re: Apple’s device surveillance plan is a threat to user privacy – and press freedom

#71

Post by intricatedetail »

kemayo wrote: post #29
intricatedetail wrote: post #25

Earlier quoted context omitted.

How would they know it is a false positive and not a CSAM image concealed in a document image that NeuralHash was able to "see"? It will be impossible to tell from a lowres picture with certainty.

My understanding is that NeuralHash is only supposed to be looking at visible pixels. Granted, I'm taking their word on that one, but I've not seen any evidence indicating that it does look for stenographically concealed images.

Not necessarily steganography. For example if the image has drastically lowered contrast, the lowres image will look like a false positive, but it will not be.

Re: Apple’s device surveillance plan is a threat to user privacy – and press freedom

#72

Post by joe_the_user »

sparker72678 wrote: post #20

I'd love to be a privacy purist on this, but the fact is that there is not going to be any going back on CSAM content scanning of images in iCloud. So either we take this approach, or end up with a worse one, like pure on-cloud scanning with no transparency whatsoever. I read the technical paper today, and this solution is super clever, well considered, and checks just about every box a crypto-solution-phile would wa…

So either we take this approach, or end up with a worse one, like pure on-cloud scanning with no transparency whatsoever.

With cloud computing, I can choose my cloud provider and I can upload encrypted files. On device scanning is evil because it's move to create a computing architecture entirely outside the user's control.

Re: Apple’s device surveillance plan is a threat to user privacy – and press freedom

#73

Post by gambiting »

tzs wrote: post #68
ohazi wrote: post #24

Earlier quoted context omitted.

Why does anyone even assume that a bad actor would need to apply pressure? These databases are unauditable by design -- all they'd need to do is hand Apple their own database of "CSAM fingerprints collected by our own local law enforcement that are more relevant in this region" (filled with political images of course), and ask Apple to apply their standard CSAM reporting rules. That's it... Tyranny complete.

Apple does not need to be able to audit the database to discover that it is not a CSAM database. Matches are reviewed by Apple before being reported to the authorities, so they would see that they are getting matches on non-CSAM material. They wouldn't necessarily be able to tell if it was a false positive matching real CSAM material or a true positive matching illegitimate material in the databases put there by a go…

That's even worse - so now apple is deciding whether to report something even if it matches the data provided by the government. It's not their role to judge the contents, only whether the match is correct or not, otherwise even with actual CSAM content, are they going to be making judgement calls? What if the system matches loli content which I imagine is in that database but legal in places? Are they going to then get the user's location(!!!!) To see whether it's legal there or not, or....guess? Or what? Because the only way to make this system work is to report every match and then let actual law enforcement figure out if it's illegal or not.

So yeah, the entire system is fucked and shouldn't exist. Apple is not law enforcement and them saying "we'll just prescreen every submission" is actually worse, not better.

Re: Apple’s device surveillance plan is a threat to user privacy – and press freedom

#74

Post by intricatedetail »

kemayo wrote: post #28
intricatedetail wrote: post #21

Earlier quoted context omitted.

How a human looking at low res CSAM matching collision picture that looks "innocent" will be able to tell for sure it is a false positive? Can they know with certainty that it is a false positive and not a manipulated real image? It seems to me that they would have to report these anyway.

Is your argument "a human review step is fundamentally a rubber-stamp that won't reject a false-positive?" Because I don't personally think that's how it'd work out, but I'll acknowledge that I might just be an optimist. (I mean, assuming that you need ~30 matches to trigger the review phase of the process, I'd think it'd be weird to a reviewer looking for child porn if you got 30 pictures of apparently-random politi…

I am trying to say it's not possible to tell with certainty from a lowres picture that you are looking at false positive. For example low contrast CSAM imposed on a document could trigger NeuralHash match but the lowres image will look like a false positive.

Re: Apple’s device surveillance plan is a threat to user privacy – and press freedom

#75

Post by joe_the_user »

sparker72678 wrote: post #67
almostdigital wrote: post #54

Earlier quoted context omitted.

I reject the premise that CSAM scanning is a solution to anything, especially if the goal is to E2EE encrypt the rest of the photos. My understanding from following this debacle is that law-enforcement don't have resources to investigate everyone who just have CSAM they focus on people who match and have other photos of novel abuse they can track down, and here they will only get an account name and a list of 30 CSAM…

I agree it’s not a real solution to a crime problem. Still seems to be the direction the political winds are blowing.

Accepting this stuff allowing yourself to blown by these evil winds, don't do that.

Part of the reason for these voluntary scans is because the state would have a difficult time implementing universal spyware on its own and it needs Apple for this. This stuff is by no means certain and how "we" helps determine whether it happens. Also, the state has secretly spied and attempted to legitimatize universal spying but we've had push back on many occasions. We should keep that up.

Re: Apple’s device surveillance plan is a threat to user privacy – and press freedom

#76

Post by kemayo »

intricatedetail wrote: post #74
kemayo wrote: post #28

Earlier quoted context omitted.

Is your argument "a human review step is fundamentally a rubber-stamp that won't reject a false-positive?" Because I don't personally think that's how it'd work out, but I'll acknowledge that I might just be an optimist. (I mean, assuming that you need ~30 matches to trigger the review phase of the process, I'd think it'd be weird to a reviewer looking for child porn if you got 30 pictures of apparently-random politi…

I am trying to say it's not possible to tell with certainty from a lowres picture that you are looking at false positive. For example low contrast CSAM imposed on a document could trigger NeuralHash match but the lowres image will look like a false positive.

For your example, wouldn't that only work to make the original source image that's polluting the CSAM database look like CSAM in lowres? The actual document-image the oppressive government is looking for that'd trigger the match wouldn't have the CSAM included.

That said, I do think it'd be nice to have a better demonstration of exactly what this "derivative" the reviewers would be looking at is. There's a lot of variations there, balancing false-positive privacy concerns, the mental health of the reviewers, potential downsampling issues, etc.

Re: Apple’s device surveillance plan is a threat to user privacy – and press freedom

#77

Post by greyface- »

35803288 wrote: post #19

This technology will soon be out of Apple’s control. Higgins correctly highlights the immense pressure Apple will get from governments and other actors to bend the technology and use it for something else than csam. It will happen, people are probably already thinking how to apply such pressure. Sooner or later Apple will cave in and they will have only themselves to blame when freedom supports in Sudan or LGBTQ acti…

Exactly. Governments tend to accept "we lack the technical capability to comply with your request" (unless said capability is legally mandated, e.g. so-called "lawful intercept"). They do not tend to accept "we possess the technical capability to comply with your request but choose not to do so".

Re: Apple’s device surveillance plan is a threat to user privacy – and press freedom

#78

Post by simondotau »

35803288 wrote: post #19

This technology will soon be out of Apple’s control. Higgins correctly highlights the immense pressure Apple will get from governments and other actors to bend the technology and use it for something else than csam. It will happen, people are probably already thinking how to apply such pressure. Sooner or later Apple will cave in and they will have only themselves to blame when freedom supports in Sudan or LGBTQ acti…

I realise that you're talking about global Governments, but when it comes to the United States, Government pressure cannot compel Apple to expand the on-device searching because that would be an unequivocal violation of the 4th Amendment of the US Constitution. Because it is a search of your private property compelled by the Government.

(After a photo is uploaded to a cloud service, a search of photos stored on servers doesn't enjoy the same 4A protection as this falls under the so-called "third party doctrine".)

(Apple searching for CSAM is also not a 4A violation because it was Apple's free choice as a private company to do so, and you will have agreed to it as part of the Terms of Service of the next version of iOS.)

Re: Apple’s device surveillance plan is a threat to user privacy – and press freedom

#79

Post by bitwise-evan »

shadowfacts wrote: post #14

While the precedent this sets is indeed concerning, the specific hypotheticals this article give are nonsensical. > an adversary could trick Apple’s algorithm into erroneously matching an existing image In which case the malicious, adversary-controlled images are sent to Apple. After which—the implication is—they can be re-obtained by... the adversary that created them. So what? An adversary could conceivably lower t…

> an adversary could trick Apple’s algorithm into erroneously matching an existing image

This is a very real, possible attack. Apple ships its CSAM model on device so any attacker can have a copy of the model. Then the attacker creates an image that triggers CSAM but looks like a panda [1]. Now the attacker sends tons of triggering photos to the unsuspecting victim, who now gets questioned by the FBI.

1: https://medium.com/@ml.at.berkeley/tricking-neural-networks-...

Re: Apple’s device surveillance plan is a threat to user privacy – and press freedom

#80

Post by OneLeggedCat »

hypothesis wrote: post #10
mrharrison wrote: post #9

There is an easy way to cast your vote for saying yes to Privacy. Turn off auto-updates and don't update to iOS 15. Spread the word.

Presumably, there will be a parallel track to stay on iOS 14 with security updates, at least for some time.

If enough people do that, then no reason that they won't just enable it in 14 too.
Post reply on HN