Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

vice.com

Post reply on HN
Print view View original

331–340 of 465 posts

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#331

Post by drvdevd »

floatingatoll wrote: post #184
epistasis wrote: post #42

What's shocking to me is how little Apple management understood of what their actions looked like. Really stunning. For a company that marketed itself as one of the few digital service providers that consumers could trust, I just don't understand how they acted this way at all. Either there will be heads rolling at management, or Apple takes a permanent hit to consumer trust.

Right now, it seems like there are two specific groups of people that are upset with Apple: Freedom evangelists (e.g. EFF) and tech futurists (e.g. HN). They're saying, essentially: "Apple does not have my permission to use my device to scan my iCloud uploads for CSAM" and "This is a slippery slope that could result in Apple enforcing thoughtcrimes" Neither of these viewpoints are particularly agreeable to the genera…

You forgot the most important point of the last 24 hours:

"Apple has created a system for detecting CSAM on local devices which has already proven vulnerable to cheap perceptual hash collision attacks. It's now highly inconceivable Apple will be able to deploy this technology as-is without having their users exploited."

In other words it's not just about privacy or thoughtcrimes anymore but should be viewed as actually dangerous to use their devices. I feel a bit dramatic even typing that out but I.. think it's true?

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#332

Post by zepto »

mightybyte wrote: post #304
zepto wrote: post #253

Earlier quoted context omitted.

> what kind of encryption can't be decrypted until some threshold has been crossed? The kind Apple has built. You should read the docs. This is literally how it works.

This is the same company that saved the disk encryption password as the password hint. You really trust them to not screw this up when the stakes are that it could ruin your life and/or land you in jail? I'm simply not ok with that.

How exactly do you imagine a bug in this will land you in jail?

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#333

Post by zepto »

atq2119 wrote: post #308
zepto wrote: post #300

Earlier quoted context omitted.

All CSAM is produced by child abuse. Everyone who shares 30 images online is a participant in child abuse, even if only by creating a demand for images. > a crime fighting strategy Is it a crime fighting strategy? I thought it was just about not making their devices and services into aids for these crimes. > The cynic in me is also wondering what fraction of people working at places like NCMEC are pedophiles. Good qu…

That's a tedious debate that has been rehashed to death. Finding somebody who has CSAM without being a child abuser doesn't save even a single child. Finding everybody who has CSAM without being a child abuser would likely save some children (drying out commercial operations, perhaps?), but is also basically impossible. And how many children can really be helped in this way? Surely the majority of child abuse happens…

> Surely the majority of child abuse happens independently of the online sharing of CSAM.

Does it? I see no reason to assume that.

> And is that really worth the suspension of important civil rights?

No civil rights are being suspended. It’s not about that at all.

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#334

Post by int_19h »

floatingatoll wrote: post #56
smoldesu wrote: post #35

Earlier quoted context omitted.

Furthermore, it may well be possible to combine that blobby grey static with another image, manipulating it's visual hash to create a "sleeper" positive. If this was possible in a week , then it's going to be very interesting to watch the technology change/evolve over the next few years.

Doing so would create a positive that still doesn't pass Apple's human visual check against the (blurred) CSAM content associated with that checksum, and if it somehow did, it would still then also have to occur at qty.30 or more, and they'd have to pass a human visual check against the (unblurred) CSAM content by one of the agencies in possession of it. It's not possible to spoof that final test unless you possess r…

At the point where it is submitted for a human visual check, your privacy has already been violated.

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#335

Post by zepto »

mschuster91 wrote: post #322
zepto wrote: post #252

Earlier quoted context omitted.

> people incompetent enough to have years old (otherwise how would it end up at NCMEC?) CSAM on their iPhones Who do you think has 30 or more CSAM images on their phone?

Pedophiles. But these people are consumers of extremely old content, which means the CSAM can won't even make a tiny dent against active, current abusers! CSAM scanning is sold as beneficial, while in reality it won't do shit while it opens dangerous precedence backdoors!

> extremely old content

Why do you assume it’s extremely old?

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#336

Post by endisneigh »

petersellers wrote: post #327
endisneigh wrote: post #320

Earlier quoted context omitted.

NCMEC is an arm of the government. Either they share voluntarily or they get subpoenaed endlessly. Why do you think all of these companies even do this? lol in any case I've given you the solution on how to use iCloud and not be scanned. Take your photos, sync your iDevice to your computer and manually upload to iCloud photos. there you go.

> NCMEC is an arm of the government. Either they share voluntarily or they get subpoenaed endlessly. A subpoena is a lot different than scanning every single photo on every user's device automatically. > Why do you think all of these companies even do this? lol Because most companies don't give a shit about privacy? And they will cave at even the slightest government pressure to do so. Honestly it's probably easier f…

If Apple really didn’t care about privacy they’d end e2ee and scan on the server side like Google and Microsoft.

> Because most companies don't give a shit about privacy? And they will cave at even the slightest government pressure to do so. Honestly it's probably easier for them that way (but worse for the consumer).

Most people don’t care. I’m giving you solutions and you’re taking about convenience how hard it is. Plugging in your phone is trivial. I assume you’re going to stick with iCloud even despite this “privacy” issue? If not, what are you switching to?

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#337

Post by nonbirithm »

mox1 wrote: post #130
helen___keller wrote: post #97

Earlier quoted context omitted.

> I just don't understand how they acted this way at all. There's a simple answer to this right? Despite everyone's reaction, Apple genuinely believe this is a novel and unique method to catch CSAM without invading people's privacy. And if you look at it from Apple's point of view that's correct: other major cloud providers catch CSAM content on their platform by inspecting every file uploaded, i.e. total invasion of…

But who was complaining about google and microsoft doing the cloud scanning? I don’t mind my one drive being scanned for “bad stuff”, I very much mind my personally owned data stores being scanned, with no opt out.

In my view, Apple is not going to take the risk of changing their mind and not implementing this new scanning feature in order to test the hypothesis that you can satisfy both the people who want to stop CSAM and the people who don't want their files scanned. And also, Apple or any other company announcing they will not scan or give up any user data means that criminals, foolish or not, will take notice and start uploading CSAM to their servers. If they get away with it, they've found their new home. That's why regulation against this kind of data exists. Possessing CSAM is still a felony.

Nobody is arguing about the legality of CSAM itself. It's an issue that is not popular to discuss, and of course being on the wrong side of it results in near-universal, justifiable derision. Stopping its spread is absolutely the right thing to do.

So at the point that companies will always be held liable for storing it, they will have to put up countermeasures of some kind or find themselves sued out of existence. Server-side scanning is one method, and Apple's on-device scanning is another.

There are certainly ways that Apple can go too far with whatever it happens to come up with as its solution to stopping CSAM, but there is still seems to have been a line behind which nobody particularly cares how the detection is implemented and life continues as usual. If Apple had chosen not to cross that line, maybe many of the arguments being made here would never have been brought up at all.

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#338

Post by bArray »

> [..] the overall system is designed to account for this to happen in general, and that the analyzed code is not the final implementation that will be used with the CSAM system itself and is instead a generic version.

I think the claim here is that you won't have access to the source images, and therefore generating collisions will be more difficult. But, if you do have access to the source images, this has been shown to be trivial. This of course doesn't stop nations states generating images that cause hash collisions, in fact they would be incentivized to do so.

I would also add that Apple are behind the curve, attempts to crack the hashing algorithm more efficiently are still ongoing: https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue...

> [..] not the final implementation [..]

Why on earth would you invite people to come and test your algorithm and then say "sure, you broke it, but it's not the real one". This kind of defeats the point and seems like some bait and switch bullshit. I suspect this is some retroactive cope from management realising they can't deploy this version and whatever they do deploy needs to be heavily modified.

> If Apple finds they are CSAM, it will report the user to law enforcement.

One statistic I want to know is: How many people already trigger this report function in the wild? Surely currently is the largest number of positives they will ever have - if it turns out to be 0% - Apple should just scrap it.

> Apple also said that after a user passes the 30 match threshold, a second non-public algorithm that runs on Apple's servers will check the results.

So to avoid reporting, simply block Apple servers? Also, security by obscurity is not security - the algorithm supposedly being private just means that its not properly tested and Apple is not held to account.

> "Apple actually designed this system so the hash function doesn't need to remain secret, as the only thing you can do with 'non-CSAM that hashes as CSAM' is annoy Apple's response team with some garbage images until they implement a filter to eliminate those garbage false positives in their analysis pipeline," Nicholas Weaver, senior researcher at the International Computer Science Institute at UC Berkeley, told Motherboard in an online chat.

No. A report could be considered 'reasonable doubt' for law enforcement to do a full search. Imagine trying to explain to a judge why your iPhone shouldn't be searched because of a false-positive CSAM hash collision because of a malicious website you visited or a text message you received.

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#339

Post by mirker »

shuckles wrote: post #314
mirker wrote: post #288

Earlier quoted context omitted.

> Apple genuinely believe this is a novel and unique method to catch CSAM without invading people's privacy What’s novel about this? The technique and issues with it are fairly obvious to anyone with experience in computer vision. It seems not too different from research from 1993 https://proceedings.neurips.cc/paper/1993/file/288cc0ff02287... The issues are also well known and encompass the whole subfield of adversa…

Presumably the system is novel enough that a very similar proposal which lacked one important insight (using perceptual hashes to turn image similarity problems into set intersection problems) was accepted to USENIX this year: https://www.usenix.org/conference/usenixsecurity21/presentat...

The privacy aspect does seem novel, and that paper evaluates the various aspects of privacy and the quality of perceptual hashes. However, anything concerning privacy (of client or server data) is orthogonal to the vision methods underpinning the system, which are well understood and are the primary cause of the post. For that matter, I don’t see how the paper’s proofs of privacy and/or formalization of the problem help with hash collisions. And that particular paper doesn’t use deep learning methods which are probably easier to attack with off-the-shelf methods.

Re: Apple defends anti-child abuse imagery tech after claims of ‘hash collisions’

#340

Post by [deleted] »

zepto wrote: post #253
pasquinelli wrote: post #192

Earlier quoted context omitted.

what kind of encryption can't be decrypted until some threshold has been crossed? maybe you mean to say that apple says they won't read it until that threshold has been crossed.

> what kind of encryption can't be decrypted until some threshold has been crossed? The kind Apple has built. You should read the docs. This is literally how it works.

[deleted]
Post reply on HN