ImageNet contains naturally occurring Apple NeuralHash collisions • Hacker News Board

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#151

Post by mrtranscendence » Thu, Aug 19, 2021, 6:34 PM UTC

stickfigure wrote: post #129

yongjik wrote: post #110

Earlier quoted context omitted.

These scenarios sound rather like "the wrong side of airlock" stories[1]. Why would China go through an elaborate scheme with fake child-porn hashes, when it can already arrest these people on made-up charges, and simply tell Apple to provide the private key for their phones, so that they can read and insert whatever real/fake evidences they want? [1] I'm stealing the expression from this excellent article: https://d…

Who said anything about fake porn hashes? China can just say to Apple: We want everyone whose phones contain XYZ subversive content. Don't like it? Don't sell phones. Apple will cave. Note that you don't even have to arrest everyone. The fear is enough to prevent thoughtcrime.

Agreed. If China can force Apple to do almost anything by threatening to ban iPhone sales, why bother with fake CSAM hashes? That just adds an extra step. It's not like the Chinese government needs to take pains to trick anyone about their attitude toward "subversive" material.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#152

Post by knodi123 » Thu, Aug 19, 2021, 6:35 PM UTC

riffraff wrote: post #86

breuleux wrote: post #67

Earlier quoted context omitted.

Presumably Apple would be afraid that, say, the EU becomes suspicious, issues a court order to obtain the hashes, notices they cannot audit the CCP hashes, pointedly asks "what is this", becomes absolutely livid that their citizens are spied on by a country that is not them, fines Apple out the wazoo, then extradites whoever is responsible and puts them in prison. I mean, China's not the only player in this. Putting…

Seems pretty trivial to have different servers sides per country, and put there a different db. EU, China, Iran, US: everyone gets to spy on their own children and forbid whatever they want.

Yeah, but if they tell us they're doing that, then it's pretty obvious what they're up to. And if they don't tell us they're doing that, but do it anyway - then they have to perpetually pay every developer involved in that upgrade enough money to keep their mouth shut indefinitely - knowing that the developers know that APPLE knows how much they'd lose in fines if they got caught. Which is an unreasonably large liability, IMO.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#153

Post by duskwuff » Thu, Aug 19, 2021, 6:35 PM UTC

pbhjpbhj wrote: post #77

stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

I think one of the other stories on this talked about "watermarking" in order to create a hash collision. So it need not be a non-CSAM image, a TLA could just alter an image to make it collide with a file they want to track, other countries would agree that file's hash should be in the hash list and bingo: Apple presumably provide the TLA with a list of devices holding that file. ?

Except that there's a threshold involved. A single matching file doesn't trigger an investigation; it takes multiple (10+, maybe more?) matches to do that.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#154

Post by stickfigure » Thu, Aug 19, 2021, 6:35 PM UTC

dylan604 wrote: post #57

stickfigure wrote: post #33

Earlier quoted context omitted.

Let's say I get you to click on a link. That link contains a thumbnail gallery of CSAM. With the right CSS, you might not even notice it, but it's in your browser's cache and on your filesystem. Lots of pictures - more than enough for your phone to snitch on you. All because you clicked on a link. Phishing attacks can now put you in prison, label you as a pedophile and sex offender, and destroy your life.

How does this fit within what Apple has stated will be the operating procedure for this scanning?

I don't know. But if the answer is "we won't snoop your browser cache", then people will quickly learn to store offending material under browser caches. Then Apple will have to start scanning browser caches. There is no policy fix for this; pedophiles aren't stupid.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#155

Post by dont__panic » Thu, Aug 19, 2021, 6:36 PM UTC

endisneigh wrote: post #137

yusefnapora wrote: post #114

Earlier quoted context omitted.

Imagine this scenario. - You receive some naughty (legal!) images of a naked young adult while flirting online and save them to your camera roll. - These images have been made to collide [1] with "well known" CSAM images obtained from the dark underbelly of the internet, on the assumption that their hashes will be contained in the encrypted database. - Apple's manual review kicks in because you have enough such image…

Your scenario makes no sense. You can just skip all of the steps and skip to "the FBI, who now have cause to turn your life upside down." If the evidence doesn't matter, they could've just reported you to the FBI for having CP, regardless of whether you have it or not and your point remains the same. Not to mention your scenario requires someone you trust trying to "get you." If that's true, then none of the other st…

If your iCloud Photo library contains enough photos to trigger a manual review + FBI report, how does the scenario make no sense?

And as far as your point about "someone you trust trying to 'get you'"... have you ever dated? Ever exchanged naughty photos with somebody (I expect this is even more popular these days among 20-somethings since covid prevented a lot of in-person hookups)? This doesn't seem crazy for a variant of catfishing. I could easily see 4chan posters doing this for fun.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#156

Post by mvanaltvorst » Thu, Aug 19, 2021, 6:36 PM UTC

matsemann wrote: post #5

> By taking advantage of the birthday paradox, and a collision search algorithm that let me search in n(log n) time instead of the naive n^2 Someone got more details on that? How does the birthday paradox come into play here?

The birthday paradox and the algorithm are not related; the birthday paradox is simply the phenomenon that even though there are several orders of magnitude more hashes than images in ImageNet, it is still likely that collisions exist.

The algorithm sounds like a simple tree search algorithm. Let's consider the naive case: traverse all images, and keep a list of hashes you have already visited. For every extra image, you have to traverse all previous n hashes you have previously computed. Naively doing this check with a for loop would take O(n) time. You have to do this traversion for every image, therefore total time complexity is O(n^2).

Fortunately, there is a faster way to check whether you have found a hash before. Imagine sorting all the previous hashes and storing them in an ordered list. A smarter algorithm would check the middle of the list, and check whether this element is higher or lower than the target hash. When your own hash is higher than the middle hash, you know that if your hash is contained within the list, it is contained in the top half. In a single iteration you have halved the search space. By repeating this over and over you can figure out if your item is contained within this list in just log_2(n) steps. This is called binary search. Some of the details are more intricate (e.g. Red-Black trees [1], where you can skip the whole sorting step) but this is the gist of it.

This all sounds way more complicated than it is in practice. In practice you would simply `include ;` and all the tree calculations are done behind the scenes. The algorithm contained within the library is clever, but the program written by the author is probably [1]: https://en.wikipedia.org/wiki/Red–black_tree

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#157

Post by dan-robertson » Thu, Aug 19, 2021, 6:36 PM UTC

It didn’t seem to take long for the weights for Apple’s network to be discovered. And I suppose they must send the banned hashes to the client for checking too. So I expect that list will be discovered and published soon too (unless they have some way to keep them secret?) I think one important question is: how reversible is Apple’s perceptual hash?

For example, my understanding of Microsoft’s PhotoDNA is that their perceptual hash has been reverse-engineered and that one could go backwards from a hash to a blurry image. But also it is very hard to get the list of PhotoDNA hashes for the NCMEC database. In other words, are Apple unintentionally releasing enough information to reconstruct a bunch of blurry CSAM?

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#158

Post by tshaddox » Thu, Aug 19, 2021, 6:36 PM UTC

stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

If the CCP says “put this arbitrary software into your next iPhone software update or we will halt all iPhone sales in China,” what do you think Apple is going to do? Isn’t the answer to both questions the same?

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#159

Post by tgsovlerkhgsel » Thu, Aug 19, 2021, 6:36 PM UTC

kempbellt wrote: post #56

flycaliguy wrote: post #49

Earlier quoted context omitted.

A corrupt government will also enjoy the “chilling effect” created by people’s fear of tainting their phone with illegal images.

I feel like a corrupt government would want people to trust their phones.

IIRC many totalitarian governments, historical and current, made/make their surveillance blatantly obvious, because the chilling effect deterring people is much more valuable than the added intelligence value from keeping people careless.

After all, they want to suppress dissent, and they can't catch and arrest everyone - it's much more effective if people don't even dare to voice their dissent.

This is also why you see people in situations like the Arab Spring use known-insecure/monitored means of communication, because they realize that the value and power that comes from communicating and finding other likeminded people is worth painting a target on yourself (because you can't succeed without it, and if you succeed, there will be too many to prosecute).

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#160

Post by foepys » Thu, Aug 19, 2021, 6:38 PM UTC

28194608 wrote: post #124

You guys are helping Apple not to fuckup their implementation by finding bugs. It would be great if these collision posts happened after they fully rolled out the update.

Once the code landed in iOS and is activated, it is too late.