ImageNet contains naturally occurring Apple NeuralHash collisions • Hacker News Board

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#161

Post by tgsovlerkhgsel » Thu, Aug 19, 2021, 6:38 PM UTC

endisneigh wrote: post #83

WesolyKubeczek wrote: post #76

Earlier quoted context omitted.

The same as what Apple is saying it's going to do , there's a difference.

If you don't trust Apple then they could've already done what you're concerned about before they announced this. I don't really get it. Either you trusted Apple before this and you continue to, or you didn't before, and continue not to. If it's the later, then you shouldn't be using Apple services.

> If you don't trust Apple then they could've already done what you're concerned about before they announced this.

Not without the risk of it being discovered (either through a leak or because someone analyzes the software on the phone), and then having a much bigger scandal on hand.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#162

Post by kemayo » Thu, Aug 19, 2021, 6:38 PM UTC

duskwuff wrote: post #153

pbhjpbhj wrote: post #77

Earlier quoted context omitted.

I think one of the other stories on this talked about "watermarking" in order to create a hash collision. So it need not be a non-CSAM image, a TLA could just alter an image to make it collide with a file they want to track, other countries would agree that file's hash should be in the hash list and bingo: Apple presumably provide the TLA with a list of devices holding that file. ?

Except that there's a threshold involved. A single matching file doesn't trigger an investigation; it takes multiple (10+, maybe more?) matches to do that.

In the interview Craig Federighi gave on Friday, he said 30.[1] I have no idea if he was just throwing a number out or if that's the actual threshold.

[1]: https://tidbits.com/2021/08/13/new-csam-detection-details-em...

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#163

Post by saithound » Thu, Aug 19, 2021, 6:39 PM UTC

Keep in mind that Apple's claimed false positive rate (one in a trillion chance of an account being flagged innocently), and the collision rate determined by Dwyer in the article, are both derived without any adversarial assumptions. Given that NeuralHash collider and similar tools already exist, the false positive rate is expected to be much much higher.

Imagine that you play a game of craps against an online casino. The casino throws a virtual six-sided die, secretly generated using Microsoft Excel's random number generator. Your job is to predict the result. If you manage to predict the result 100 times in a row, you win and the casino will pay you $1000000000000 (one trillion dollars). If you ever fail to predict the result of a throw, the game is over, you lose and you pay the casino $1 (one dollar).

In an ordinary, non-adversarial context, the probability that you win the game is much less than one in one trillion, so this game is very safe for the casino. But this number is very misleading: it's based on naive assumptions that are completely meaningless in an adversarial context. If your adversary has a decent knowledge of mathematics at the high school level, the serial correlation in Excel's generator comes into play, and the relevant probability is no longer one in one trillion. The relevant number is 1/216 instead! When faced with a class of adversarial math majors, a casino that offers this game will promptly go bankrupt. With Apple's CSAM detection, you get to be that casino.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#164

Post by orhmeh09 » Thu, Aug 19, 2021, 6:39 PM UTC

willis936 wrote: post #59

orhmeh09 wrote: post #54

Earlier quoted context omitted.

Could you provide specific evidence that China has and would do this? I’ve a hard time recalling any specific cases. Maybe nation-states do this kind of thing, but I’m only aware of the countless times the United States has done this. What’s the recent history?

Do you have access to this webpage? https://en.wikipedia.org/wiki/1989_Tiananmen_Square_protests

What does this have to do with forcing corporations’ hands?

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#165

Post by jb1991 » Thu, Aug 19, 2021, 6:39 PM UTC

> Apple's NeuralHash perceptual hash function performs its job better than I expected and the false-positive rate on pairs of ImageNet images is plausibly similar to what Apple found between their 100M test images and the unknown number of NCMEC CSAM hashes.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#166

Post by yeldarb » Thu, Aug 19, 2021, 6:39 PM UTC

dan-robertson wrote: post #157

It didn’t seem to take long for the weights for Apple’s network to be discovered. And I suppose they must send the banned hashes to the client for checking too. So I expect that list will be discovered and published soon too (unless they have some way to keep them secret?) I think one important question is: how reversible is Apple’s perceptual hash? For example, my understanding of Microsoft’s PhotoDNA is that their…

It doesn't seem like it; check out the adversarially constructed images here. They don't look anything like the original despite perfectly matching the NeuralHash: https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue...

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#167

Post by nonbirithm » Thu, Aug 19, 2021, 6:40 PM UTC

Dah00n wrote: post #53

rootusrootus wrote: post #38

Earlier quoted context omitted.

after ~30 matches, only the matching images are passed on to a human for visual verification. only images uploaded to iCloud are subject to matching

If they pass CSAM verfied by hash on to human verification inside Apple they break the law. Not even the FBI are allowed to do that. Only NMCEC is an allowed recipient by US federal law.

> Not even the FBI are allowed to do that.

The FBI was given clearance to take over a website serving CSAM in order to catch more users of the site. As such, the FBI has technically distributed CSAM in the past.

https://www.dallasnews.com/news/crime/2017/01/17/the-fbi-ran...

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#168

Post by tyingq » Thu, Aug 19, 2021, 6:40 PM UTC

stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

>(like a foreign government)

It seems like it wouldn't take that. If you can generate a colliding pair of images, you could probably create a pair where one of the images might get attention with child porn groups and thus, shared around enough to end up in the CSAM database. And where the other was innocuous.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#169

Post by orhmeh09 » Thu, Aug 19, 2021, 6:40 PM UTC

tyingq wrote: post #61

orhmeh09 wrote: post #54

Earlier quoted context omitted.

Could you provide specific evidence that China has and would do this? I’ve a hard time recalling any specific cases. Maybe nation-states do this kind of thing, but I’m only aware of the countless times the United States has done this. What’s the recent history?

That China would use heavy handed tactics to coerce a US tech company? Google for "Operation Aurora" Also see: https://www.reuters.com/article/us-china-apple-icloud-insigh... for a very similar situation to the one described up-thread.

Assuming you are American — where do you think your iCloud keys are stored? You do know Apple cooperates with US LE and intelligence? This is a nothing hamburger.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#170

Post by bsdetector » Thu, Aug 19, 2021, 6:41 PM UTC

stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

> If the CCP says "put these hashes in your database or we will halt all iPhone sales in China", what do you think Apple is going to do?

Or maybe China already said "put in this CSAM check or you can't make or sell phones in China".

Since Apple's position is contrary to their previous privacy policy and doesn't seem to make a lot of sense, it's quite possible extortion already happened (and not necessarily by China).