Live data from Hacker News

ImageNet contains naturally occurring Apple NeuralHash collisions

blog.roboflow.com

271–280 of 530 posts

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#271

Earlier quoted context omitted.

> If the CCP says "put these hashes in your database or we will halt all iPhone sales in China", what do you think Apple is going to do? Or maybe China already said "put in this CSAM check or you can't make or sell phones in China". Since Apple's position is contrary to their previous privacy policy and doesn't seem to make a lot of sense, it's quite possible extortion already happened (and not necessarily by China).

I'd honestly believe such pressure came from US domestic intelligence or law enforcement agencies just as easily as from China.

It wouldn't specifically be from domestic intelligence, it would be from a powerful member of Congress with a relationship to Apple (specifically the board/management), acting as a go-between that would try to politically lay out the situation for them.

Hey Apple, we can either turn up the anti-trust heat by a lot, or we can turn it down, which is it going to be? Except it would be couched in intellectually dishonest language meant to preserve a veneer that the US Government isn't a violent, quasi-psychotic bully ready to bash your face in if you don't do what they're asking.

The interactions with intelligence about the new program would begin after they acquiesce to going along.

It's too easy. There's an extraordinary amount of wilful naivety in the US about the nature of the government and its frequent power abuses (what it's willing to do), despite the rather comically massive demonstration of said abuses spanning the entire post WW2 era. Every time it happens the wilfully naive crowd feigns surprise.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#272
post #25

Earlier quoted context omitted.

What if those two countries can be Poland and Hungary? These two countries have been passing lots of laws to ostracize and criminalize pro-LGBT content and are friendly to each other.

> These two countries have been passing lots of laws to criminalize pro-LGBT content What do you mean I live in one of those countries and I'd want to be aware

https://googlethatforyou.com?q=poland%20anti-lgbt

but actually this is a good starting point: https://en.wikipedia.org/wiki/LGBT_rights_in_Poland

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#273

It doesn’t matter if there are collisions if the two images don’t actually look the same. Do people honestly believe a single CSAM flag from an “innocent” image is going to result in someone going to prison in America? PhotoDNA has existed for over a decade doing the same thing with no instances that I have heard of. If some corrupt government wants to get you they don’t need this. They can just unilaterally say you’…

At least part of the concern is that a hash collision is basically "cause" for Apple to then dump and begin manually (like, with humans) reviewing the contents of your device, all of which will be happening behind the closed doors of a private corporation, outside of any of the usual oversight or innocent-presumption mechanisms that come from it happening through the courts. That, combined with a (pretty reasonable)…

What they would be reviewing would be scaled version of the specific photos that triggered the hash alert. It’s not a broad fishing expedition. There is no mechanism to start browsing the photos on your phone.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#274
post #70

> In order to test things, I decided to search the publicly available ImageNet dataset for collisions between semantically different images. I generated NeuralHashes for all 1.43 million images and searched for organic collisions. By taking advantage of the birthday paradox, and a collision search algorithm that let me search in n(log n) time instead of the naive n^2, I was able to compare the NeuralHashes of over 2…

>2 in 2 trillion image pairs Reporting the collision rate per image pair feels misleading. What you really want to know is the number of false positives per image in the relevant set, not image pair, as that's the figure that indicates how frequently you'll hit a false positive.

In fact, I'd argue that the collision rate per image pair is overestimating the collision rate. It's the flip side of the birthday paradox. We don't care that any two images have the same hash, we care about any image having the same hash as one in the set that we're testing against.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#275

Earlier quoted context omitted.

Agreed. If China can force Apple to do almost anything by threatening to ban iPhone sales, why bother with fake CSAM hashes? That just adds an extra step. It's not like the Chinese government needs to take pains to trick anyone about their attitude toward "subversive" material.

Exactly. Apple can already ship literally any conceivable software to iPhones. Do people really think their plan was to sneak functionality into this update and then update the CSAM database later, and they would have gotten away with it if it weren't for the brilliant privacy advocates pointing out that this CSAM database could be changed over time? That's pretty ludicrous. If the Chinese government wanted to (and t…

So your argument boils down to since Apple can already install software without us knowing, we shouldn't worry about a new client-side system that makes it substantially easier for nation states to abuse? I don't find that argument the least bit compelling.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#276

Earlier quoted context omitted.

The same as what Apple is saying it's going to do , there's a difference.

Man, wouldn't you love to be the developer who gets assigned the feature to commit these horrible secret privacy violations with deeply evil ethical problems? You don't even have to implement the feature. All you need is really good proof that you were asked to, and now your job at Apple is secure, along with a huge raise, for years if not for life. Something commensurate with what you and they both know they would l…

> possible EU fines

Until the EU makes it a legal requirement. Which they're getting close to

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#277
post #213
post #194

Earlier quoted context omitted.

> The end result is that some peon at Apple has to look at the images and mark them as not CSAM Btw, this reminded me of a podcast about FB's group to do just this. Because it negatively impacted the mental health of those FB employees, they farmed it out to the contractors in other countries. There were interviews with women in the Philippines, and it was having the same impact there.

It’s quite possible that you implied this, but I think that the true positives are the ones that had a mental health toll.

That's correct. It wasn't just CSAM. The described images were sickening.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#278

Earlier quoted context omitted.

The same as what Apple is saying it's going to do , there's a difference.

Man, wouldn't you love to be the developer who gets assigned the feature to commit these horrible secret privacy violations with deeply evil ethical problems? You don't even have to implement the feature. All you need is really good proof that you were asked to, and now your job at Apple is secure, along with a huge raise, for years if not for life. Something commensurate with what you and they both know they would l…

> wouldn't you love to be the developer who gets assigned the feature

Also, they'd probably not use a Cupertino developer. I'm sure a dev in a nation with a lot less rights is easier for this sort of work. Find a nation where the protections for employees are worse and good jobs harder to find.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#279

Earlier quoted context omitted.

What probability do you ascribe to that reviewer clicking the wrong button, be it out of habit/zoning out (because the system usually shows them true positives), cheating (always clicking "yes" because it's usually correct and allows them to get paid without having to look at horrible images all day), mistake, wrong instructions (e.g. thinking that all images of children, or all porn including adult porn, should be f…

Even if Apple’s manual review fails, there’s still the NCMEC’s review. There are several layers before anything goes to law enforcement.

And presumably, at some point even if it makes it to a trial, the accused would be able to point to the flagged images to show that they aren't actually child porn.

And a country in which this isn't possible, is likely one that is going to ruin people's lives just fine without Apple.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#280
post #9

> In order to test things, I decided to search the publicly available ImageNet dataset for collisions between semantically different images. I generated NeuralHashes for all 1.43 million images and searched for organic collisions. By taking advantage of the birthday paradox, and a collision search algorithm that let me search in n(log n) time instead of the naive n^2, I was able to compare the NeuralHashes of over 2…

ImageNet is a very well-known data set. Are we sure apple didn't test on it when designing this algorithm?

Nah, the consensus online seems to be that Apple hires naive, inept script kiddies and that any rando on GitHub can prove without question that Apple’s solution is flawed.
Post reply on HN