Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ImageNet contains naturally occurring Apple NeuralHash collisions

blog.roboflow.com

Post reply on HN
Print view View original

261–270 of 530 posts

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#261

Post by foota »

nomel wrote: post #216
mrtranscendence wrote: post #179

Earlier quoted context omitted.

Why would anyone bother with such an attack? The end result is that some peon at Apple has to look at the images and mark them as not CSAM. You've cost someone a bit of privacy, but that's it.

Why is this question being downvoted? I too would like to know what this attack achieves. From what I see, the end result of false flagging is either someone has CSAM in iCloud and you push them over the threshold that results in reporting and prosecution, or there is no CASM, so the reviewer sees all of the hash collision images, including those that are natural. Is the problem that an attacker can force natural has…

The idea I've heard is that images could be generated that are sexual in nature but that have been altered to match a CSAM hash, making a tricky situation.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#262

Post by saithound »

robertoandred wrote: post #256
saithound wrote: post #232

Earlier quoted context omitted.

Okay, let's play peon. Here are three perfectly legal and work-safe thumbnails of a famous singer: https://imgur.com/a/j40fMex . The singer is underage in precisely one of the three photos. Can you decide which one? If your account has a large number of safety vouchers that trigger a CSAM match, then Apple will gather enough fragments to reassemble a secret key X (unique to your device) which they can use to decrypt…

Apple can only ever see the visual derivatives in vouchers of images that match CSAM hashes, not vouchers of all your images.

Yep. I'm aware of this, and it doesn't affect the point I was making, but it's worth pointing out. I made an edit to the text to make this explicit.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#263

Post by kevin_thibedeau »

slg wrote: post #75
foerbert wrote: post #69

Earlier quoted context omitted.

The difference is the ease with which they can demur. Before, it would be a whole heck of a lot of new, additional work. They also have the problem of actually introducing it without being noticed, or having to come up with some cover for the new behavior. Now? Well now it's real simple. It will even conveniently not expose the actual images it's checking for. Apple now has significantly less ability to rationally re…

As far as I'm aware, this system is not new. It is only moving from the cloud to the local device. If the cloud was already compromised, which it seems like it would be in your logic since all the same reasoning applies, I don't understand the complaints about it moving locally. In my mind there are two possible ways to view this. We could trust Apple last month and we can trust them today. We couldn't trust Apple la…

They are crossing a property boundary:

You know food poisoning is dangerous and you'll be safer with a food taster to make sure nothing you eat is spoiled. I'll just help myself to your domicile and eat your food to make sure it's all safe. I already made a copy of your keys to let myself in. It's for your own good.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#264

Post by vineyardmike »

mrtranscendence wrote: post #179
saithound wrote: post #163

Keep in mind that Apple's claimed false positive rate (one in a trillion chance of an account being flagged innocently), and the collision rate determined by Dwyer in the article, are both derived without any adversarial assumptions. Given that NeuralHash collider and similar tools already exist, the false positive rate is expected to be much much higher. Imagine that you play a game of craps against an online casino…

Why would anyone bother with such an attack? The end result is that some peon at Apple has to look at the images and mark them as not CSAM. You've cost someone a bit of privacy, but that's it.

> The end result is that some peon at Apple has to look at the images and mark them as not CSAM.

As others said, if the non-csam looks sexual at all, they'll probably get flagged for post-apple review.

Beyond that, it doesn't seem to be in apple's interest to be conservative in flagging. An employee reviewer's best interest is to minimize false negatives not false positives.

As many mentioned, even an investigation can have horrible affects on some (innocent) person's life. I would not be shocked to learn that some crafty individual working at "meme factories" creating intentional collisions with distributed images just for "fun" - and politically motivated attacks seem plausible (eg. make liberal political memes flag CSAM).

Then there are targeted motives for an attack. Have a journalist you want to attack or find a reason to warrant? Find them on dating app and send them nudes with CSAM collisions. Or any number of other targetted attacks against them.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#265

Post by bawolff »

endisneigh wrote: post #28
bawolff wrote: post #24

Earlier quoted context omitted.

In a proposed system that has yet to be deployed?

This has already existed in PhotoDNA since 2008. This is not new

But that's deployed in a very different way which makes the concerns being discussed much less likely to happen.

Specificly the person doing the scanning already has access to photos and can double check the results without having to sieze the device, a rather public process.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#266

Post by robertoandred »

Youden wrote: post #107
endisneigh wrote: post #8

It doesn’t matter if there are collisions if the two images don’t actually look the same. Do people honestly believe a single CSAM flag from an “innocent” image is going to result in someone going to prison in America? PhotoDNA has existed for over a decade doing the same thing with no instances that I have heard of. If some corrupt government wants to get you they don’t need this. They can just unilaterally say you’…

> It doesn’t matter if there are collisions if the two images don’t actually look the same. Is that really true? My understanding is that the manual reviewers at Apple only see some kind of low-resolution proxy, not the full-resolution image. I'd also be shocked if the human reviewers were shown the original, actually CP image, to compare to. Given that, it's not necessary to produce an actual visual match, it's just…

Why would you save all of the almost-CSAM pics the attacker sends you to your photo library?

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#267

Post by robertoandred »

toby- wrote: post #246
endisneigh wrote: post #28

Earlier quoted context omitted.

This has already existed in PhotoDNA since 2008. This is not new

Examples of on-device scanning via PhotoDNA?

People’s photo libraries are scanned. The result is the same.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#268

Post by dhosek »

breuleux wrote: post #103
riffraff wrote: post #86

Earlier quoted context omitted.

Seems pretty trivial to have different servers sides per country, and put there a different db. EU, China, Iran, US: everyone gets to spy on their own children and forbid whatever they want.

The db is encrypted and uploaded to user devices. If each country gets a different db, the payload will be different in each country, which does not make sense if it's all supposed to be CSAM. So Apple would likely just say "these were mandated by the US government for US citizens," punting the ball in their court, unless they are forbidden to say so, in which case they'll say nothing, but we all know what it means.…

>That's when you know you should change phones

You'd have to switch to a dumb phone. Assuming you can find one that works on contemporary networks.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#269

Post by godelski »

adventured wrote: post #250
godelski wrote: post #238

Earlier quoted context omitted.

Or why not "Hey Vietnam, Pakistan, Russia, etc put these hashes into your database please and thanks." I mean the CCP has allies that are also authoritarian. Why would they have to threaten Apple directly? This is also how you get past the Apple human verification. Just pay those Apple workers to click confirm.

> Why would they have to threaten Apple directly? They'd do it directly because it's expedient and useful. If you're operating such a sprawling authoritarian regime, it's important to occasionally make a show of your power and control, lest anyone forget. The CCP isn't afraid of Apple, Apple is afraid of the CCP. Lately the CCP has been on a rather showy demonstration of its total control. If you're them it's useful…

I'm just saying that there is another avenue. To be clear, this isn't a "vs" situation. It means that they have multiple avenues.

To also clarify, the avenue of extortion isn't open to every country. But the avenue I presented is as long as that country has an ally. I'm not aware of any country not having an ally, so I presume that this avenue is pretty much open to any country.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#270

Post by IncRnd »

azinman2 wrote: post #234
sillysaurusx wrote: post #111

Earlier quoted context omitted.

This isn’t true. The db is blinded. We have no way of knowing what’s in it. It would be trivial to have the same payload on each device, and extract different answers using the matching server side db which varies by country. Perhaps not trivial, but just short.

What do you mean blinded? It’s already been promised there will be a way to verify the hash db on your own device.

There are many functions to which cryptographic blinding is applied, but they each rely upon multiple parties to compute the function in question. In that way, the input and output are blinded to a single party.
Post reply on HN