Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ImageNet contains naturally occurring Apple NeuralHash collisions

blog.roboflow.com

Post reply on HN
Print view View original

221–230 of 530 posts

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#221

Post by breuleux »

sillysaurusx wrote: post #111
breuleux wrote: post #103

Earlier quoted context omitted.

The db is encrypted and uploaded to user devices. If each country gets a different db, the payload will be different in each country, which does not make sense if it's all supposed to be CSAM. So Apple would likely just say "these were mandated by the US government for US citizens," punting the ball in their court, unless they are forbidden to say so, in which case they'll say nothing, but we all know what it means.…

This isn’t true. The db is blinded. We have no way of knowing what’s in it. It would be trivial to have the same payload on each device, and extract different answers using the matching server side db which varies by country. Perhaps not trivial, but just short.

I feel that it's the kind of scheme that requires too much cooperation from too many people and organizations with conflicting incentives. It's possible some countries would not want the hashes from certain other governments in the db at all. And then what? I may be wrong, but I also believe we can know how many hashes are in the db, which means that if it contains extra hashes from dozens of governments, it would become suspiciously large relative to how many CSAM images we know exist. Furthermore, in this scenario the db cannot be auditable, so the scheme falls apart as soon as some rogue judge decides to order an audit.

I honestly don't think Apple wants to deal with any of that crap and that they would rather silently can the system and do exactly what everybody else does than place themselves in the line of fire when their own unique trademark system is being abused.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#222

Post by shuckles »

stjohnswarts wrote: post #138
shuckles wrote: post #36

Earlier quoted context omitted.

That is literally the status quo with every cloud service. Apple, unlike the others, has said that they will evaluate you on the basis of what’s included in the associated data of your safety voucher, and you can inspect those contents because they’re shipped in the client. Facebook, for all I know, might be calculating a child predator likelihood score on my account based on how often I look up my middle school ex-g…

I don't understand technie people on HN being okay with apple breaching the spirit of the 4th amendment and becoming the FBI agent in your phone. Scanning the stuff in the cloud is one thing but this is crossing a line. I am shedding all my apple hardware over it. If you want to trust them fine but one day it will bite you on the ass.

For ideological consistency, are you dumping every service provider that scans the contents of your account and reports offending data to law enforcement?

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#223

Post by sangnoir »

mrtranscendence wrote: post #151
stickfigure wrote: post #129

Earlier quoted context omitted.

Who said anything about fake porn hashes? China can just say to Apple: We want everyone whose phones contain XYZ subversive content. Don't like it? Don't sell phones. Apple will cave. Note that you don't even have to arrest everyone. The fear is enough to prevent thoughtcrime.

Agreed. If China can force Apple to do almost anything by threatening to ban iPhone sales, why bother with fake CSAM hashes? That just adds an extra step. It's not like the Chinese government needs to take pains to trick anyone about their attitude toward "subversive" material.

> why bother with fake CSAM hashes?

Because Apple has already built that functionality, and it exists? What alternative dragnet currently exists to identify iOS users who possess certain images? This would be code reuse.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#224

Post by ixfo »

yongjik wrote: post #110
stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

These scenarios sound rather like "the wrong side of airlock" stories[1]. Why would China go through an elaborate scheme with fake child-porn hashes, when it can already arrest these people on made-up charges, and simply tell Apple to provide the private key for their phones, so that they can read and insert whatever real/fake evidences they want? [1] I'm stealing the expression from this excellent article: https://d…

Just so you know, the original source of the expression would be The Hitchhiker's Guide to the Galaxy.

VOGON GUARD: I’ll mention what you said to my aunt.

[Airlock door closes and locks]

FORD: Potentially bright lad, I thought.

ARTHUR: We’re trapped now, aren’t we?

FORD: Er… Yes, we’re trapped.

ARTHUR: Well didn’t you think of anything?

FORD: Oh Yes.

ARTHUR: Yes?

FORD: But, unfortunately, it rather involved being on the other side of the airtight hatchway they’ve just sealed behind us.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#225

Post by giantrobot »

yongjik wrote: post #110
stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

These scenarios sound rather like "the wrong side of airlock" stories[1]. Why would China go through an elaborate scheme with fake child-porn hashes, when it can already arrest these people on made-up charges, and simply tell Apple to provide the private key for their phones, so that they can read and insert whatever real/fake evidences they want? [1] I'm stealing the expression from this excellent article: https://d…

China or any government adding collisions would be to use Apple's system as a dragnet to find users possessing the offending images.

The way it would work is the government in question would submit legitimate CSAM but modified to produce a collision with a government target image. Looking at the raw image (or a derivative) a reviewer at Apple or ICMEC would see a CSAM image. The algorithm would see the anti-government image. So Apple scans Chinese (or whoever) citizens libraries, finds "CSAM" and reports them to ICMEC which then reports them to the government in question.

Every repressive government and some notionally liberal governments will eventually do this. It likely is already happening with existing PhotoDNA systems. The difference is that's being used by explicit sharing services where Apple's new system will search for any photo in a user's library regardless of it being "shared" explicitly.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#226

Post by mrtranscendence »

oconnore wrote: post #139

If it's so easy to modify NeuralHashes, won't "CSAM networks" just rotate the hashes of their "collections"? If you can make an innocent picture collide with a CSAM picture, presumably you can also edit a CSAM picture to have a random hash not in the database?

Through chatting and reading threads I get the impression that many folks think that pedophiles are all smart, extremely tech savvy, operating in networks that share information and collaborate to fool law enforcement. There may be some of that, but there are also plenty of Joe Schmoes who download or trade CSAM material without being especially clever about it. I remember when I was a teenager downloading porn I found more than a few probably-illegal photos just by accident*. If you want to find such images you can, it doesn't take an organized "network".

* edit - this was 25 years ago, admittedly, and much of what I downloaded came from randos on AOL. Things might be different now.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#227

Post by willis936 »

orhmeh09 wrote: post #164
willis936 wrote: post #59

Earlier quoted context omitted.

Do you have access to this webpage? https://en.wikipedia.org/wiki/1989_Tiananmen_Square_protests

What does this have to do with forcing corporations’ hands?

If you have access: scroll to the "Contemporary Issues" section. Further reading linked in that section:

https://en.wikipedia.org/wiki/Censorship_in_China

https://en.wikipedia.org/wiki/Overseas_censorship_of_Chinese...

https://en.wikipedia.org/wiki/Internet_censorship_in_the_Peo...

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#228

Post by giantrobot »

tshaddox wrote: post #212
mrtranscendence wrote: post #151

Earlier quoted context omitted.

Agreed. If China can force Apple to do almost anything by threatening to ban iPhone sales, why bother with fake CSAM hashes? That just adds an extra step. It's not like the Chinese government needs to take pains to trick anyone about their attitude toward "subversive" material.

Exactly. Apple can already ship literally any conceivable software to iPhones. Do people really think their plan was to sneak functionality into this update and then update the CSAM database later, and they would have gotten away with it if it weren't for the brilliant privacy advocates pointing out that this CSAM database could be changed over time? That's pretty ludicrous. If the Chinese government wanted to (and t…

> they could literally just tell Apple to issue a software update that streams all desired private data to Chinese government servers.

Uh...this already happened. [0][1]

[0] https://www.macrumors.com/2021/05/17/apple-security-compromi...

[1] https://support.apple.com/en-us/HT208351

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#229

Post by theluketaylor »

zug_zug wrote: post #184
endisneigh wrote: post #8

It doesn’t matter if there are collisions if the two images don’t actually look the same. Do people honestly believe a single CSAM flag from an “innocent” image is going to result in someone going to prison in America? PhotoDNA has existed for over a decade doing the same thing with no instances that I have heard of. If some corrupt government wants to get you they don’t need this. They can just unilaterally say you’…

And what if the FBI demands a list of all Apple users who have matched even 1 CSAM photo for their own private watchlist?

There are a lot of really valid criticisms of Apple plan here, but Apple has gone out of their way to prevent that exact case. Apple is using secret splitting to make sure they cannot decode the CSAM ticket until the threshold is reached. Devices also produce some synthetic matches to prevent themselves Apple (or anyone else) inferring a pre-threshold count based on the number of vouchers.

https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...

Threshold Secret Sharing Synthetic Match Vouchers

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#230

Post by kccqzy »

dan-robertson wrote: post #157

It didn’t seem to take long for the weights for Apple’s network to be discovered. And I suppose they must send the banned hashes to the client for checking too. So I expect that list will be discovered and published soon too (unless they have some way to keep them secret?) I think one important question is: how reversible is Apple’s perceptual hash? For example, my understanding of Microsoft’s PhotoDNA is that their…

> And I suppose they must send the banned hashes to the client for checking too.

They absolutely do not. They use Private Set Intersection to achieve that.

Post reply on HN