Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ImageNet contains naturally occurring Apple NeuralHash collisions

blog.roboflow.com

Post reply on HN
Print view View original

51–60 of 530 posts

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#51

Post by tehnub »

ulzeraj wrote: post #25
criticaltinker wrote: post #4

Earlier quoted context omitted.

The OP mentions that two countries have to agree to add a file to the list, but your concern is definitely valid: > Perhaps the most concerning part of the whole scheme is the database itself. Since the original images are (understandably) not available for inspection, it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights adv…

What if those two countries can be Poland and Hungary? These two countries have been passing lots of laws to ostracize and criminalize pro-LGBT content and are friendly to each other.

Apple did mention in their security thread model document [0] this:

  Apple will also refuse all
  requests to instruct human reviewers to file reports for 
  anything other than CSAM materials for accounts that exceed 
  the match threshold.
[0]: https://www.apple.com/child-safety/pdf/Security_Threat_Model...

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#52

Post by slg »

stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

I would expect Apple to say the same thing if the CCP proposed a system of scanning devices last month. I fail to see how this system changes the calculus for how Apple will deal with authoritarian governments.

If Apple could stand up to them before this system, why can't they stand up to them with this system?

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#53

Post by Dah00n »

rootusrootus wrote: post #38
pizzaknife wrote: post #21

the world of forensics is something im not familiar with, a couple questions: - When a hash matches (correctly or incorrectly), how is said image reported? is the matching image passed on to a human to verify or? - what is the survey size / who(m) are subject to this CSAM net?

after ~30 matches, only the matching images are passed on to a human for visual verification. only images uploaded to iCloud are subject to matching

If they pass CSAM verfied by hash on to human verification inside Apple they break the law. Not even the FBI are allowed to do that. Only NMCEC is an allowed recipient by US federal law.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#54

Post by orhmeh09 »

stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

Could you provide specific evidence that China has and would do this? I’ve a hard time recalling any specific cases. Maybe nation-states do this kind of thing, but I’m only aware of the countless times the United States has done this. What’s the recent history?

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#55

Post by tibbar »

tehnub wrote: post #47

> In order to test things, I decided to search the publicly available ImageNet dataset for collisions between semantically different images. I generated NeuralHashes for all 1.43 million images and searched for organic collisions. By taking advantage of the birthday paradox, and a collision search algorithm that let me search in n(log n) time instead of the naive n^2, I was able to compare the NeuralHashes of over 2…

1) the point of the birthday paradox is that even if two elements of a set are unlikely to overlap, in a large set it is much more likely than perhaps intuitive that some pair of elements overlap. 2) I’m assuming he did something like putting all the hashes in a list and sorting them, which is at least better than looking at each pair. As you say, it’s not optimal and also doesn’t seem particularly worth including in the otherwise very interesting post

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#56

Post by kempbellt »

flycaliguy wrote: post #49
zbobet2012 wrote: post #16

Earlier quoted context omitted.

That's not the route in which this will be exploited, and used at scale. A corrupt government has to know they want you "to just get you". Instead they will embed a collision in anti-government meme. That collision will flag you, and now they know you harbor doubts and will come get you. This is why it's a privacy concern. It's no the tech (like you said photo dna's been about forver), it's the scanning of the phone.

A corrupt government will also enjoy the “chilling effect” created by people’s fear of tainting their phone with illegal images.

I feel like a corrupt government would want people to trust their phones.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#57

Post by dylan604 »

stickfigure wrote: post #33
endisneigh wrote: post #8

It doesn’t matter if there are collisions if the two images don’t actually look the same. Do people honestly believe a single CSAM flag from an “innocent” image is going to result in someone going to prison in America? PhotoDNA has existed for over a decade doing the same thing with no instances that I have heard of. If some corrupt government wants to get you they don’t need this. They can just unilaterally say you’…

Let's say I get you to click on a link. That link contains a thumbnail gallery of CSAM. With the right CSS, you might not even notice it, but it's in your browser's cache and on your filesystem. Lots of pictures - more than enough for your phone to snitch on you. All because you clicked on a link. Phishing attacks can now put you in prison, label you as a pedophile and sex offender, and destroy your life.

How does this fit within what Apple has stated will be the operating procedure for this scanning?

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#58

Post by waz0wski »

orhmeh09 wrote: post #54
stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

Could you provide specific evidence that China has and would do this? I’ve a hard time recalling any specific cases. Maybe nation-states do this kind of thing, but I’m only aware of the countless times the United States has done this. What’s the recent history?

https://www.reuters.com/article/us-china-apple-icloud-insigh...

https://support.apple.com/en-us/HT208351

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#59

Post by willis936 »

orhmeh09 wrote: post #54
stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

Could you provide specific evidence that China has and would do this? I’ve a hard time recalling any specific cases. Maybe nation-states do this kind of thing, but I’m only aware of the countless times the United States has done this. What’s the recent history?

Do you have access to this webpage?

https://en.wikipedia.org/wiki/1989_Tiananmen_Square_protests

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#60

Post by tines »

slg wrote: post #52
stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

I would expect Apple to say the same thing if the CCP proposed a system of scanning devices last month. I fail to see how this system changes the calculus for how Apple will deal with authoritarian governments. If Apple could stand up to them before this system, why can't they stand up to them with this system?

Tin-foil hat time: Who's to say that they could stand up to them before this system? The system itself could have been proposed by the CCP in the first place. I'll take my hat off now.
Post reply on HN