ImageNet contains naturally occurring Apple NeuralHash collisions • Hacker News Board

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#291

Post by throwaway672000 » Thu, Aug 19, 2021, 8:12 PM UTC

mrtranscendence wrote: post #251

saithound wrote: post #232

Earlier quoted context omitted.

Okay, let's play peon. Here are three perfectly legal and work-safe thumbnails of a famous singer: https://imgur.com/a/j40fMex . The singer is underage in precisely one of the three photos. Can you decide which one? If your account has a large number of safety vouchers that trigger a CSAM match, then Apple will gather enough fragments to reassemble a secret key X (unique to your device) which they can use to decrypt…

Fair enough. I suppose it's true that you could create a colliding sexually explicit image where age is indeterminate, and the reviewer may not realize it isn't a match. > Given the ability to produce hash collisions, an adversary can easily generate photos that fail this visual inspection as well. Apple could easily fix this by also showing a low-res version of the CSAM image that was collided with, but I'll grant t…

Won't enough images be real matches for them to be looking at it (in low res) for most of their work day?

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#292

Post by slg » Thu, Aug 19, 2021, 8:12 PM UTC

colpabar wrote: post #193

slg wrote: post #113

Earlier quoted context omitted.

>But how long before it scans everything, and there's no way to opt out? Do we think this is detectable? If yes, then why worry about it if we will know when this switch is made? If not, why did we trust Apple that this wasn't happening already? That is the primary thing I don't understand, this fear rests on an assumption that Apple is a combination of both honest and corrupted. If they are honest, we have no reason…

It feels like you're viewing this as a purely hypothetical question and ignoring reality. No company is 100% good or bad, and it doesn't make any sense to force all possible interpretations into good/bad. > If not, why did we trust Apple that this wasn't happening already? I do not trust Apple. I don't really trust any major tech company, because they put profit first, and everything else comes second. I believe that…

>I do not trust Apple. I don't really trust any major tech company, because they put profit first, and everything else comes second.

Then you should have never been using a closed system like Apple in which they had control over every aspect of it. That is my fundamental point. I'm not saying you should trust Apple. I am saying this shouldn't have changed your opinion on Apple.

>So that we can all get used to it, and not make a big fuss when google announces android will do the same thing. It's much easier to do things without needing to keep them a secret. This is in no way only about apple, they're just breaking the ice so to speak.

I just need more evidence before I believe a global conspiracy that requires the coordination between both adversarial governments and direct business competitors.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#293

Post by nicce » Thu, Aug 19, 2021, 8:14 PM UTC

tablespoon wrote: post #147

burkaman wrote: post #117

Earlier quoted context omitted.

Because they don't know who to arrest yet. The idea isn't to fabricate a charge, it's to locate people sharing politically sensitive images that the government hasn't already identified.

> Because they don't know who to arrest yet. The idea isn't to fabricate a charge, it's to locate people sharing politically sensitive images that the government hasn't already identified. And maybe even identify avenues for sharing that they haven't already identified and monitored/controlled (e.g. some encrypted chat app they haven't blocked yet).

China does not really need Apple to do much. They already make installation of some apps mandatory by law. Also, some communication must be done with WeChat and so on. They have pretty good grip already.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#294

Post by [deleted] » Thu, Aug 19, 2021, 8:15 PM UTC

tshaddox wrote: post #158

stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

If the CCP says “put this arbitrary software into your next iPhone software update or we will halt all iPhone sales in China,” what do you think Apple is going to do? Isn’t the answer to both questions the same?

[deleted]

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#295

Post by sennight » Thu, Aug 19, 2021, 8:16 PM UTC

kmbfjr wrote: post #255

sennight wrote: post #136

Earlier quoted context omitted.

We are talking about how everyone who gave Apple money now has a potential probable cause vector that they didn't before. Everyone running the software is a suspect by default. Ask black Americans how they feel about setting the bar low for probable cause. "Following the 2004 Madrid train bombings, fingerprints on a bag containing detonating devices were found by Spanish authorities. The Spanish National Police share…

Reading about incidences such as this has made me think critically about all cloud services in the United States, and the conclusion is simply not to use them. Sure, the probability is lower than getting struck by lightning. I certainly don't play in the rain and I won't be using cloud services where I'm exposed to this kind of nonsense with the FBI.

> Sure, the probability is lower than getting struck by lightning.

I don't think anyone can actually know that, because I don't think statistics are kept on how often these sort of dragnet programs result in civil liberty violations and secret grand juries. That should be the more immediate concern, because after that point you are depending on the goodwill of prosecutors... which is a super bad idea.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#296

Post by conradev » Thu, Aug 19, 2021, 8:17 PM UTC

azinman2 wrote: post #249

sillysaurusx wrote: post #240

Earlier quoted context omitted.

It’s blinded in the cryptographic sense. It’s a specific term. I would go into detail, but . Suffice to say, unless you provide proof, I am reasonably confident there’s no way to verify the hash db doesn’t contain extra hashes other than the CSAM hashes provided by the US government. But I’ve been wrong many times before.

Well first of all, it's not provided by the US government. It's a non-profit, and Apple has already said they're going to look for another db from another nation and only included hashes that are the union of the two to prevent exactly this kind of attack. If what you mean by blinded is that you don't know what the source image is for the hash, that's true. Otherwise Apple would just be putting a database of child po…

I would just read the document explaining how this works (see "Matching-Database Setup"): https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

At no point is anyone besides Apple able to view any NeuralHash hashes from the CSAM database. You can verify the database is the same on all iPhones, but you are not able to look at any of the hashes.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#297

Post by jonplackett » Thu, Aug 19, 2021, 8:18 PM UTC

tshaddox wrote: post #158

stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

If the CCP says “put this arbitrary software into your next iPhone software update or we will halt all iPhone sales in China,” what do you think Apple is going to do? Isn’t the answer to both questions the same?

If they do one of those, it will be obvious it happened, because people can at least reverse engineer iOS and see that it’s different.

If they add some new hashes I presume that would be harder to spot and isn’t going to be advertised

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#298

Post by dwighttk » Thu, Aug 19, 2021, 8:19 PM UTC

stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

Apple doesn’t control the database

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#299

Post by saithound » Thu, Aug 19, 2021, 8:20 PM UTC

ChrisKnott wrote: post #286

saithound wrote: post #232

Earlier quoted context omitted.

Okay, let's play peon. Here are three perfectly legal and work-safe thumbnails of a famous singer: https://imgur.com/a/j40fMex . The singer is underage in precisely one of the three photos. Can you decide which one? If your account has a large number of safety vouchers that trigger a CSAM match, then Apple will gather enough fragments to reassemble a secret key X (unique to your device) which they can use to decrypt…

You're adding quite a lot of technobabble gloss to an "attack vector" that boils down to "people can send you images that are visually indistinguishable from known CSAM". Guess what, they can already do this but worse by just sending you actual illegal images of 17.9 year olds. While it would be bad to be subjected to such an attack, and there is a small chance it would lead to some kind of interaction with law enfor…

I suggest you reread the comment, because "people can send you images that are visually indistinguishable from known CSAM" is not what is being said at all. Where did you even get that from?

The point is precisely that people can become victims of various new attacks, without ever touching photos that are actual "known CSAM". For Christ's sake, half the comments here are about how adversaries can create and spread political memes that trigger automated CSAM filters on people's phones just to "pwn the libz".

> Guess what, they can already do this but worse by just sending you actual illegal images of 17.9 year olds.

No, this misses the point completely. You cannot easily trigger any automated systems merely by taking photos of 17.9 year olds and sending them to people. E.g. your own photos are not in the NCMEC databases, and you'd have to reveal your own illegal activities to get them in there. You (or malicious political organizations) especially cannot attack and expose "wrongthinking" groups of people by sending them photos of 17.9 year olds.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#300

Post by Bjartr » Thu, Aug 19, 2021, 8:22 PM UTC

breuleux wrote: post #67

stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

Presumably Apple would be afraid that, say, the EU becomes suspicious, issues a court order to obtain the hashes, notices they cannot audit the CCP hashes, pointedly asks "what is this", becomes absolutely livid that their citizens are spied on by a country that is not them, fines Apple out the wazoo, then extradites whoever is responsible and puts them in prison. I mean, China's not the only player in this. Putting…

> citizens are spied on by a country that is not them

I thought countries often have under the table agreements with one another to explicitly spy on each others citizens, since its illegal for the country to spy on its own citizens. It's illegal for the other country too, but it's a lot easier to turn a blind eye to it.