Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ImageNet contains naturally occurring Apple NeuralHash collisions

blog.roboflow.com

Post reply on HN
Print view View original

281–290 of 530 posts

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#281

Post by IncRnd »

ixfo wrote: post #224
yongjik wrote: post #110

Earlier quoted context omitted.

These scenarios sound rather like "the wrong side of airlock" stories[1]. Why would China go through an elaborate scheme with fake child-porn hashes, when it can already arrest these people on made-up charges, and simply tell Apple to provide the private key for their phones, so that they can read and insert whatever real/fake evidences they want? [1] I'm stealing the expression from this excellent article: https://d…

Just so you know, the original source of the expression would be The Hitchhiker's Guide to the Galaxy. VOGON GUARD: I’ll mention what you said to my aunt. [Airlock door closes and locks] FORD: Potentially bright lad, I thought. ARTHUR: We’re trapped now, aren’t we? FORD: Er… Yes, we’re trapped. ARTHUR: Well didn’t you think of anything? FORD: Oh Yes. ARTHUR: Yes? FORD: But, unfortunately, it rather involved being on…

Maybe, but it probably stretches farther back than that, maybe even to before sliced bread or cool beans. Ten years before The Hitchhiker's Guide there was a robot, HAL, who woudn't open the airlock for a particular astronaut.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#282

Post by stefan_ »

nomel wrote: post #216
mrtranscendence wrote: post #179

Earlier quoted context omitted.

Why would anyone bother with such an attack? The end result is that some peon at Apple has to look at the images and mark them as not CSAM. You've cost someone a bit of privacy, but that's it.

Why is this question being downvoted? I too would like to know what this attack achieves. From what I see, the end result of false flagging is either someone has CSAM in iCloud and you push them over the threshold that results in reporting and prosecution, or there is no CASM, so the reviewer sees all of the hash collision images, including those that are natural. Is the problem that an attacker can force natural has…

You are one underpaid random guy in India looking at CSAM all day clicking the wrong button away from a raid of your home and the end of your life as you know it.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#283

Post by noptd »

stjohnswarts wrote: post #122
axelf4 wrote: post #116

Earlier quoted context omitted.

That is not a good comparison. The extra hashes would help China find out about more borderline citizens than it otherwise would have.

Have we established that a US NGO is accepting "CSAM" hashes from China or that they are cooperating with them at all? That seems unlikely and Apple hasn't yet announced plans with how they're going to scan phones in China, I mean wouldn't China just demand outright to have full scanning capabilities of anything on the phone since you don't have any protection at all from that in China?

Sure, but Apple receives far less backlash if the system is applied to all phones and under the guise of "save the children". This would allow Apple to accommodate any nation state's image scanning requirements, which guarantees their continued operation in said markets.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#284

Post by [deleted] »

mrtranscendence wrote: post #197
zug_zug wrote: post #184

Earlier quoted context omitted.

And what if the FBI demands a list of all Apple users who have matched even 1 CSAM photo for their own private watchlist?

And what if the FBI demands that Apple leadership genuflect toward an oil painting of J Edgar Hoover? The FBI can demand all sorts of things. In this case, Apple isn't tracking collisions as small as 1 photo.

[deleted]

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#285

Post by noptd »

tshaddox wrote: post #158
stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

If the CCP says “put this arbitrary software into your next iPhone software update or we will halt all iPhone sales in China,” what do you think Apple is going to do? Isn’t the answer to both questions the same?

So we should simply accept systems with a high potential for abuse because of the possibility that something bad is already being done?

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#286

Post by ChrisKnott »

saithound wrote: post #232
mrtranscendence wrote: post #179

Earlier quoted context omitted.

Why would anyone bother with such an attack? The end result is that some peon at Apple has to look at the images and mark them as not CSAM. You've cost someone a bit of privacy, but that's it.

Okay, let's play peon. Here are three perfectly legal and work-safe thumbnails of a famous singer: https://imgur.com/a/j40fMex . The singer is underage in precisely one of the three photos. Can you decide which one? If your account has a large number of safety vouchers that trigger a CSAM match, then Apple will gather enough fragments to reassemble a secret key X (unique to your device) which they can use to decrypt…

You're adding quite a lot of technobabble gloss to an "attack vector" that boils down to "people can send you images that are visually indistinguishable from known CSAM".

Guess what, they can already do this but worse by just sending you actual illegal images of 17.9 year olds.

While it would be bad to be subjected to such an attack, and there is a small chance it would lead to some kind of interaction with law enforcement, the outcomes you present are just scaremongering and not reasonable.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#287

Post by vlovich123 »

mrtranscendence wrote: post #251
saithound wrote: post #232

Earlier quoted context omitted.

Okay, let's play peon. Here are three perfectly legal and work-safe thumbnails of a famous singer: https://imgur.com/a/j40fMex . The singer is underage in precisely one of the three photos. Can you decide which one? If your account has a large number of safety vouchers that trigger a CSAM match, then Apple will gather enough fragments to reassemble a secret key X (unique to your device) which they can use to decrypt…

Fair enough. I suppose it's true that you could create a colliding sexually explicit image where age is indeterminate, and the reviewer may not realize it isn't a match. > Given the ability to produce hash collisions, an adversary can easily generate photos that fail this visual inspection as well. Apple could easily fix this by also showing a low-res version of the CSAM image that was collided with, but I'll grant t…

The problem is that it is a scaled low-res version. There are well publicized attacks[1] showing you can completely change the contents of the image post scaling. There's also the added problem that if the scaled down image is small, even without the attack, it's impossible to make a reasonable human judgement call (as OP points out).

The problem isn't CSAM scanning in principle. The problem is that the shift to the client & the various privacy-preserving steps Apple is attempting to make is actually making the actions taken in response to a match different in a concerning way. One big problem isn't the cases where the authorities should investigate*, but that a malicious actor can act surreptitiously and leave behind almost no footprint of the attack. Given SWATting is a real thing, imagine how it plays out if child pornography is a thing. From the authorities perspective SWATting is low incidence & not that big a deal. Very different perspective on the victim side though.

[1] https://embracethered.com/blog/posts/2020/husky-ai-image-res...

* One could argue about the civil liberties aspect & the fact that having CSAM images is not the same as actually abusing children. However, among the general population that line of reasoning just gets you dismissed as supporting child abuse & is only starting to become acknowledged in the psychiatry community.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#288

Post by sennight »

dane-pgp wrote: post #248
sennight wrote: post #136

Earlier quoted context omitted.

We are talking about how everyone who gave Apple money now has a potential probable cause vector that they didn't before. Everyone running the software is a suspect by default. Ask black Americans how they feel about setting the bar low for probable cause. "Following the 2004 Madrid train bombings, fingerprints on a bag containing detonating devices were found by Spanish authorities. The Spanish National Police share…

> '100% verified' Just reading those words is rage-inducing, but I'm grateful to have learnt this example of government lying. I feel like it should become an expression that societies teach to their children to warn them about abuses of power. Other mottoes synonymous with government deception and corruption come to mind, but at the risk of being too controversial I will share only their initials and dates: "SAARH"…

Because I delight in delivering bad news, I'll point out that the real takeaway shouldn't be that federal LEOs regularly lie (though, they do) - it is that they are permitted to lie convincingly through handwavy technical means. All these tools are designed to give them permission to totally destroy your life. I'm aware of only two geewiz CSI methods that are actually founded in science: cryptography (this neural crap doesn't qualify) and DNA. Unlike fingerprints and bitemark analysis, those two tools were invented outside of law enforcement - instead of being purpose built for prosecution. Anybody doubting that should look into the history of the polygraph and its continued use in the face of evidence demonstrating how useless it is in the pursuit of truth... which begs the question: if they aren't interested in the truth, what are they doing?

https://en.wikipedia.org/wiki/Polygraph

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#289

Post by javierbyte »

slg wrote: post #134
yeldarb wrote: post #92

Earlier quoted context omitted.

Their reported collision rate was against their CSAM hash database[1]. > In Apple’s tests against 100 million non-CSAM images, it encountered 3 false positives when compared against NCMEC’s database. In a separate test of 500,000 adult pornography images matched against NCMEC’s database, it found no false positives. [1] https://tidbits.com/2021/08/13/new-csam-detection-details-em...

I don't follow the point you are making here. The goal of the algorithm is to match images so I would expect similar collision rates regardless of what image was matched against what set of hashes. The exact rate of collision will obviously vary slightly.

I think the point is that if Apple did optimize the algorithm using ImageNet then they and we are seeing the same best case scenario.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#290

Post by zug_zug »

theluketaylor wrote: post #229
zug_zug wrote: post #184

Earlier quoted context omitted.

And what if the FBI demands a list of all Apple users who have matched even 1 CSAM photo for their own private watchlist?

There are a lot of really valid criticisms of Apple plan here, but Apple has gone out of their way to prevent that exact case. Apple is using secret splitting to make sure they cannot decode the CSAM ticket until the threshold is reached. Devices also produce some synthetic matches to prevent themselves Apple (or anyone else) inferring a pre-threshold count based on the number of vouchers. https://www.apple.com/child…

Thanks. You're right. Very sophisticated system. Still find it morally repugnant, but technologically it is fairly thorough.
Post reply on HN