Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ImageNet contains naturally occurring Apple NeuralHash collisions

blog.roboflow.com

Post reply on HN
Print view View original

501–510 of 530 posts

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#501

Post by skygazer »

sennight wrote: post #482
skygazer wrote: post #452

Earlier quoted context omitted.

my aim was to point out that the above reverenced "image scaling attack" is easily protected against, because it is fragile to alternate scaling methods -- it breaks if you don't use the scaling algorithm the attacker planned for, and there exist secure scaling algorithms that are immune. [0] Since defeating the image scaling attack is trivial, it means that, if it is addressed, the thumbnail will always resemble the…

> However, that confusing image would almost certainly not also fool Apple's unspecified secondary server-side hashing algorithm, as referenced on page 13 of Apple's Security Threat Model Review... Uh, on what timescale? If you mean "tomorrow" then sure, if you mean "for years" - then no. They're relying on the second perceptual hashing algorithm to remain a secret, which is insanely foolish. Just based on what I kno…

Yeah, I don’t have that answer, of course. But nothing prevents them from changing that secondary algorithm yearly, or at whatever rate the CSAM database owners would tolerate full rehashing, or chaining together multiple hashes. They can literally tune it to whatever arbitrary false positive rate they want. Although, not knowing any better, I would guess that they would just use Microsoft’s PhotoDNA hash unchanged, and just keep it under wraps, since I think that’s what they already use for iCloud email attachment scanning. PhotoDNA just does a scaled down, black and white edge/intensity gradient comparison, and not a neural net feature detection. I would think using a completely different technology would make the pair of algorithms extremely robust taken together, but that’s not my field.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#502

Post by teekert »

mrtranscendence wrote: post #179
saithound wrote: post #163

Keep in mind that Apple's claimed false positive rate (one in a trillion chance of an account being flagged innocently), and the collision rate determined by Dwyer in the article, are both derived without any adversarial assumptions. Given that NeuralHash collider and similar tools already exist, the false positive rate is expected to be much much higher. Imagine that you play a game of craps against an online casino…

Why would anyone bother with such an attack? The end result is that some peon at Apple has to look at the images and mark them as not CSAM. You've cost someone a bit of privacy, but that's it.

Some dudette/dude is going to look at my personal pictures every now and then? What if they are of my naked children and what if that person is a csam interested person? And she/he takes a picture of the screen? Ugh it feels so bad!

I don’t want there to be a chance some person is going to look at my pics!

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#503

Post by blub »

int_19h wrote: post #346
rovr138 wrote: post #339

Earlier quoted context omitted.

On top of that, what happens if a court/government orders them to give them all the current data about people with matches, regardless of the 30 matches. They can't say if it is or not a match so they have to go after the individuals. Is that enough evidence for a warrant? Someone in the court thinks it's true and can't prosecute?, oh, it got leaked . -- Not every country has the same protections about innocent until…

More broadly speaking, every part of this scheme that is currently an arbitrary Apple decision (and not a technological limitation), can easily become an arbitrary government decision. And yes, it's true that the governments could always mandate such scanning before. The difference is that it'll be much harder politically for Apple to push back against tweaks to the scheme (such as lowering the bar for manual review…

5 million pounds can ensure a comfortable life everywhere in the world today and was likely worth much more in the past.

Assuming said socialite was not in a committed relationship, why would they not take that money for what must be 30m of effort which may actually be pleasant?

5 pounds on the other hand is not only a small amount of money, but it’s also insulting to ask somebody that’s not a prostitute to sleep with one for such a pittance.

Fictional Churchill was acting like an asshole and the fictional socialite was acting rationally. She only should have replied instead “X million pounds is the best I can offer, but I should certainly hope you are good in bed Mr. Churchill”.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#504

Post by bluebit »

spideymans wrote: post #334
nomel wrote: post #236

Earlier quoted context omitted.

I think you misunderstand the reporting process. If the threshold is passed, Apple reports the images to NCMEC for review. NCMEC reports it to authorities. So, this would require the failure of three organizations.

It would also require the failure of both the prosecutor and the judiciary to recognize the images as non-CSAM.

I honestly don't know what people are arguing here - in what universe would a non-CSAM image that has been incorrectly flagged ruin a person's life?

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#505

Post by dgellow »

FabHK wrote: post #385
dgellow wrote: post #121

Earlier quoted context omitted.

I think that you overestimate the EU reaction. Every few years we learn that our Europeans leaders and some citizens have been again spied by foreign powers, such as the US, and absolutely nothing ever happened.

> I think that you overestimate the EU reaction. Great (depressing) Twitter account: Is EU Concerned? @ISEUConcerned Very, deeply, strongly, seriously, gravely, extremely, unprecedentedly https://twitter.com/ISEUConcerned

Thanks, that's a really funny one :)

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#506

Post by saithound »

ec109685 wrote: post #492
saithound wrote: post #489

Earlier quoted context omitted.

We have no reason to believe that Apple's second, secret perceptual hash provides any meaningful protection against such attacks. At best, we can hope that it'll allow early detection of attacks in a few cases, but chances are that's the best it can do. We might not ever learn: Apple now has a very strong incentive not to admit to any evidence of abuse or to any faults in their algorithm. (Sorry, this is going to be…

The first half of your post is predicated on it being likely the noise added to generate hash A using the NeuralHash is likely to produce a specific hash B with some unknown perceptual hashing function (which they specifically call out [1] as independent of the NeuralHash function precisely because they don’t want to make this easy, so speculating it might be the NeuralHash run again is incorrect). Hash A is generate…

> Until you can show an example or paper where the same adversarial image generates a specific hash value for two unrelated perceptual hash functions, with one being hidden, it is not right to predict a high likelihood of that first scenario being possible.

"There is no paper attacking ROT13 done twice, therefore it must be secure". Usually, it's on the one proposing the protocol to make a case for its security. Doubly so when it's supposed to last a long time, a lot of people are interested in attacking it, and successful attacks can put people in harm's way.

You know what, if you think that this is difficult, feel free to pick an existing perceptual hash function H, cough up some money, and we'll announce a modest prize (say $4000) on HN for the first person to have a working collision attack for NeuralHash+H. H will run on a scaled-down thumbnail, and we'll keep the precise identity of the algorithm secret. If the challenge gets any traction, but nobody succeeds within 40 days, I'll pay you $4000 for your effort. If you're right, this should be easy money. (cf SHA1, which lasted 22 years)

Heck If Apple claims that this is difficult (afaict they don't, it would be unwise), they might even join in with their own preimage challenge for $$$. It'd be a no-brainer, a simple and cheap way of generating good publicity.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#507

Post by sennight »

skygazer wrote: post #501
sennight wrote: post #482

Earlier quoted context omitted.

> However, that confusing image would almost certainly not also fool Apple's unspecified secondary server-side hashing algorithm, as referenced on page 13 of Apple's Security Threat Model Review... Uh, on what timescale? If you mean "tomorrow" then sure, if you mean "for years" - then no. They're relying on the second perceptual hashing algorithm to remain a secret, which is insanely foolish. Just based on what I kno…

Yeah, I don’t have that answer, of course. But nothing prevents them from changing that secondary algorithm yearly, or at whatever rate the CSAM database owners would tolerate full rehashing, or chaining together multiple hashes. They can literally tune it to whatever arbitrary false positive rate they want. Although, not knowing any better, I would guess that they would just use Microsoft’s PhotoDNA hash unchanged,…

> But nothing prevents them from changing...

While there may not be an immovable obstacle standing between them and a complete recataloging, there are a lot of factors that would strongly disincentivise it. Chief among them being the fact that the project is a already a radioactive cost center - and unless they plan on switching industries and giving Blue Coat a run for its money, it always will be.

> ...chaining together multiple hashes.

That would be the lazy programmer way to do it that would very likely result in a situation where correlation starts popping up - that is why DBAs weren't advised to do some whacky md5/sha1 mashup that avoids requiring every user rekey in the wake of a digest bump up.

> ...I would guess that they would just use Microsoft’s PhotoDNA hash unchanged...

That is a reasonable guess, because that is what all the NGOs have been using - IWF being one of the more notorious. That would be bad news though, for anyone expected the thumbnail perceptual hashing step to provide meaningful protection.

> I would think using a completely different technology would make the pair of algorithms extremely robust...

Nope - which is why you don't see hybrid cryptographic algorithms. Also, if they are using PhotoDNA on their verification step then they actually implemented the thing totally backwards... because the high-pass filter approach makes it resistant to the hash length extension attacks that are imperceivable to humans. That counts for nothing by the time the first algorithm has been fooled by an extension attack (and this neural thing is definitely vulnerable to it), because the attacker would already be selecting for a thumbnail image that would fool a human in the second step - and PhotoDNA would be looking for the exact same thing that a human would: points of contrast.

BTW, PhotoDNA is a black box with no outside scrutiny to speak of - you can count on one hand the number of papers where it is even mentioned (and only ever in passing).

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#508

Post by sandos »

giantrobot wrote: post #254
mrtranscendence wrote: post #179

Earlier quoted context omitted.

Why would anyone bother with such an attack? The end result is that some peon at Apple has to look at the images and mark them as not CSAM. You've cost someone a bit of privacy, but that's it.

It's entirely possible to alter an image such that its raw form looks different from its scaled form [0]. A government or just well resourced group can take a legitimate CSAM image and modify it such that when scaled for use in the perceptual algorithm(s) it changes to be some politically sensitive image. Upon review it'll look like CSAM so off it goes to reporting agencies. Because the perceptual hash algorithms are…

That has to be scaling where you dont properly get rid of all the high-frequency bits of information before scaling, no?

I have a hard time seeing this happen with a "perfect" scaling algoritm in the spatial domain.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#509

Post by fluidcruft »

I guess I'd care if any of the collisions weren't of things that I can vaguely see as looking similar. It's not like a nail or a ski looks like child porn. I'm actually impressed. Who cares that they collide. The manufactured collisions are obviously a problem though if I understand correctly that it means CP could be hidden by making it collide with well-known (and likely whitelisted) images.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#510

Post by minitoar »

robertoandred wrote: post #267
toby- wrote: post #246

Earlier quoted context omitted.

Examples of on-device scanning via PhotoDNA?

People’s photo libraries are scanned. The result is the same.

Only if you’re putting your stuff in the cloud. There’s a big difference between my files on my computer and my files on someone else’s computer. Or at least there should be, IMO.
Post reply on HN