Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

121–130 of 725 posts

Re: Hash collision in Apple NeuralHash model

#121

Post by goohle »

sitharus wrote: post #56
the8472 wrote: post #27

Earlier quoted context omitted.

> 7. Apple reviewer confuses a featureless blob of gray with CSAM material, several times A better collision won't be a grey blob, it'll take some photoshopped and downscaled picture of a kid and massage the least significant bits until it is a collision. https://openai.com/blog/adversarial-example-research/

So the person would have to accept and save an image that when looks enough like CSAM to confuse a reviewer…

Like this one: https://flickr.com/photos/tonysanchez/2526100122

Re: Hash collision in Apple NeuralHash model

#122

Post by shapefrog »

icelancer wrote: post #59
halflings wrote: post #8

Apple's scheme includes operators manually verifying a low-res version of each image matching CSAM databases before any intervention. Of course, grey noise will never pass for CSAM and will fail that step. The fact that you can randomly manipulate random noise until it matches the hash of an arbitrary image is not surprising. The real challenge is generating a real image that could be mistaken for CSAM at low res + i…

> Of course, grey noise will never pass for CSAM and will fail that step. Never? You sure that one or more human operators will never make this mistake, dooming someone's life / causing them immense pain?

I can guarantee nobody will see the inside of a courtroom, on charges of possession and distribution of child porn for possessing multiple images of grey noise (unless there is some steganography going on).

Re: Hash collision in Apple NeuralHash model

#123

Post by Retr0id »

wisienkas wrote: post #90

Not knowing too much of the NeuralHash model, but why are they using MD5 hash, they are known to have many collisions. We don't use MD5 for private/public keys for the same reason

What makes you think they're using MD5 anywhere?

Even if they were, it wouldn't matter, because NeuralHash is non-cryptographic by design.

Re: Hash collision in Apple NeuralHash model

#124

Post by zimpenfish »

basilgohar wrote: post #116
zimpenfish wrote: post #88

Earlier quoted context omitted.

> but the image has already been sent to Apple, where a reviewer marked it as CP No, the images are only decryptable after a threshold (which appears to be about 30) is breached. If you've received 30 pieces of CSAM from WhatsApp contacts without blocking them and/or stopping WhatsApp from automatically saving to iCloud, I gotta say, it's on you at that point.

Just a side point, a single WhatsApp message can contain up to 30 images. 30 is the literal max of a single message. So ONE MESSAGE could theoretically contain enough images to trip this threshold.

> a single WhatsApp message can contain up to 30 images

A fair point, yes, and somewhat scuppering towards my argument.

Re: Hash collision in Apple NeuralHash model

#125

Post by Ensorceled »

brigandish wrote: post #49
ptidhomme wrote: post #5

Yes, just like rape accusations. It doesn't matter that you prove it was false afterwards. Edit : well that was a hint to Assange of course. Probably not true in general. So yes, I mean false accusations.

I think you may have attracted less downvotes if the phrasing was changed to "Yes, just like false accusations of rape , it doesn't matter that you prove it was false afterwards." I also think that those downvoting you might've applied the principle of charity and taken the best interpretation of what you've written or at least ask .

> I also think that those downvoting you might've applied the principle of charity and taken the best interpretation of what you've written or at least ask.

There are far too many people who assume/assert false accusations of rape are the norm for the principle of charity to apply here.

Re: Hash collision in Apple NeuralHash model

#126

Post by sodality2 »

nyuszika7h wrote: post #66
WA wrote: post #53

Earlier quoted context omitted.

Is it so hard to understand? Some people don’t use cloud storage for precisely the reason that the photos are not encrypted. Now they can’t even use their phone for storing photos. The thing with "only when iCloud is enabled" is only for now. It’s trivial to make Scanning all photos default in a future version.

That would require a software update and would definitely not go unnoticed. Would you rather they implement scanning on server side and never be able to enable end-to-end encryption for iCloud Photos? I imagine that might be the end goal, otherwise I don't see why they wouldn't have just done it on server side. Sure, this system still has the potential to be abused, but if I had to choose between "end-to-end encrypte…

False dichotomy. You should UNEQUIVOCALLY not be FORCED to choose one of those. You shouldn't choose one of those at all. I do not need to be treated like a criminal.

Facebook et al scans server side because they have liability. Tell me why apple thinks they need to scan locally? They dont have liability, which is even worse. It means they're doing it for other reasons. Plenty of people read that as "wow apple is so kindhearted they did this without the business incentive". And you know what? They didn't. They might even have good intentions. But, as the saying goes, the path to hell is paved with good intentions.

Re: Hash collision in Apple NeuralHash model

#127

Post by short_sells_poo »

UncleMeat wrote: post #109

Why is this meaningfully different than, say, what Google Photos has been doing for years? If you can get rooting malware on the target device then you could 1. Produce actual CSAM rather than a hash collision 2. Produce lots of it 3. Sync it with Google Photos This attack has been available for many years and does not need convoluted steps like hash collisions if you have the means to control somebody's phone with a…

Google was not standing on a pedestal preaching privacy. In contrast Apple was trying to appear as the privacy conscious hardware/software vendor. To now implement such a blatantly obvious stepping stone to dragnet surveillance of actual devices is such a hypocritical move that it beggars belief.

Ignoring the whataboutism in your question, we know that privacy once lost is practically impossible to get back. Once the genie is out of the bottle and Apple is doing on device scanning, what's to stop 3 letter agencies and governments around the world to start demanding ever more access? Because that's exactly what's going to happen. "Oh it's not a big deal, we just need slightly more access than we have now."

10 years down the line, we'd have people's phones datamining every piece of info they have and silently reporting to an unknowable set of entities. All in the name of fighting crime.

There needs to be pushback on this crap. Every single one of these attempts must be met with absolute and unconditional refusal. Mere inaction means things will inevitably and invariably get worse over time.

Re: Hash collision in Apple NeuralHash model

#129

Post by short_sells_poo »

reuben_scratton wrote: post #15
user-the-name wrote: post #9

Earlier quoted context omitted.

It can't. No actions are taken on hashes alone. The procedure is, if an account uploads some number of images with matching hashes, those images are verified by a human. This can attack that system itself, though, by overloading those humans with too much work looking at random noise, but that requires quite a large organised effort. It also requires getting a hold of actual blacklisted hashes, which I doubt anyone h…

"Verified by a human" - and that human will be an overworked, underpaid, overseas subcontractor who may well have an incentive to mash the "Confirm match" button from time to time to improve his performance.

Beyond that, are we seriously going to ignore the fact that eventually this system will share the private photos of someone's naked kid with some random subcontractor? How is that even remotely OK? Or that it will find and share actual CSAM with said subcontractor?

Re: Hash collision in Apple NeuralHash model

#130

Post by Ensorceled »

sneak wrote: post #52
brigandish wrote: post #49

Earlier quoted context omitted.

I think you may have attracted less downvotes if the phrasing was changed to "Yes, just like false accusations of rape , it doesn't matter that you prove it was false afterwards." I also think that those downvoting you might've applied the principle of charity and taken the best interpretation of what you've written or at least ask .

All criminal accusations, including true ones , should be treated as false until the accused is proven guilty. This is a fundamental tenet of human rights in western, small-l liberal free societies. The fact that this is controversial these days is literally insane to me. The consequences of throwing this fundamental system out the window is that you get the sort of nonsense that happened with Assange, where he was l…

> All criminal accusations, including true ones, should be treated as false until the accused is proven guilty.

No, they need to be treated as unproven, a very critical difference.

Just to be clear, witness testimony, including testimony FROM THE VICTIM, is evidence of the crime. Just for some reason, in rape cases, we go all wonky with this principle.

Post reply on HN