Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ImageNet contains naturally occurring Apple NeuralHash collisions

blog.roboflow.com

Post reply on HN
Print view View original

71–80 of 530 posts

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#71

Post by vorpalhex »

slg wrote: post #52
stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

I would expect Apple to say the same thing if the CCP proposed a system of scanning devices last month. I fail to see how this system changes the calculus for how Apple will deal with authoritarian governments. If Apple could stand up to them before this system, why can't they stand up to them with this system?

Err, Apple allows the CCP to scan iCloud already, and did so willingly and actively.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#72

Post by sweezyjeezy »

tehnub wrote: post #47

> In order to test things, I decided to search the publicly available ImageNet dataset for collisions between semantically different images. I generated NeuralHashes for all 1.43 million images and searched for organic collisions. By taking advantage of the birthday paradox, and a collision search algorithm that let me search in n(log n) time instead of the naive n^2, I was able to compare the NeuralHashes of over 2…

By "taking advantage of the Birthday Paradox" - he means that even though the chance of two random images colliding is ~ 1/1 trillion, if you have a set of size ~ sqrt(1 trillion) you have a good chance of having a collision amongst all pairs.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#73

Post by dotcommand »

waz0wski wrote: post #58
orhmeh09 wrote: post #54

Earlier quoted context omitted.

Could you provide specific evidence that China has and would do this? I’ve a hard time recalling any specific cases. Maybe nation-states do this kind of thing, but I’m only aware of the countless times the United States has done this. What’s the recent history?

https://www.reuters.com/article/us-china-apple-icloud-insigh... https://support.apple.com/en-us/HT208351

> https://www.reuters.com/article/us-china-apple-icloud-insigh...

Apple has to store data of chinese citizens in china? And Apple has to adhere to chinese laws in china? How insane.

It's crazy how deluded the "CCP crowd" are. Apparently, the "CCP crowd" thinks companies are allowed to do business in another country and not abide by their laws.

Are you going to go insane since the EU requires tech companies to store EU citizens data within the EU?

https://blogs.microsoft.com/eupolicy/2021/05/06/eu-data-boun...

You would think most countries would demand their citizens data be stored locally.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#74

Post by benjaminpv »

kempbellt wrote: post #56
flycaliguy wrote: post #49

Earlier quoted context omitted.

A corrupt government will also enjoy the “chilling effect” created by people’s fear of tainting their phone with illegal images.

I feel like a corrupt government would want people to trust their phones.

Alternatively, a corrupt government might want folks to distrust their mass market phone such that they can have an individual come along and offer them a 'completely secure and private' alternative[1].

[1]: https://www.pcmag.com/news/fbi-sold-criminals-fake-encrypted...

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#75

Post by slg »

foerbert wrote: post #69
slg wrote: post #52

Earlier quoted context omitted.

I would expect Apple to say the same thing if the CCP proposed a system of scanning devices last month. I fail to see how this system changes the calculus for how Apple will deal with authoritarian governments. If Apple could stand up to them before this system, why can't they stand up to them with this system?

The difference is the ease with which they can demur. Before, it would be a whole heck of a lot of new, additional work. They also have the problem of actually introducing it without being noticed, or having to come up with some cover for the new behavior. Now? Well now it's real simple. It will even conveniently not expose the actual images it's checking for. Apple now has significantly less ability to rationally re…

As far as I'm aware, this system is not new. It is only moving from the cloud to the local device. If the cloud was already compromised, which it seems like it would be in your logic since all the same reasoning applies, I don't understand the complaints about it moving locally.

In my mind there are two possible ways to view this.

We could trust Apple last month and we can trust them today.

We couldn't trust Apple last month and we can't trust them today.

I don't understand the mindset that we could trust Apple last month and we can't trust them today.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#76

Post by WesolyKubeczek »

endisneigh wrote: post #43
biesnecker wrote: post #41

Earlier quoted context omitted.

They're not actively scanning your phone, they're actively scanning files you send them.

That’s the same as what Apple is going to do - scan right before sending to iCloud

The same as what Apple is saying it's going to do, there's a difference.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#77

Post by pbhjpbhj »

stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

I think one of the other stories on this talked about "watermarking" in order to create a hash collision. So it need not be a non-CSAM image, a TLA could just alter an image to make it collide with a file they want to track, other countries would agree that file's hash should be in the hash list and bingo: Apple presumably provide the TLA with a list of devices holding that file.

?

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#78

Post by rootusrootus »

Dah00n wrote: post #53
rootusrootus wrote: post #38

Earlier quoted context omitted.

after ~30 matches, only the matching images are passed on to a human for visual verification. only images uploaded to iCloud are subject to matching

If they pass CSAM verfied by hash on to human verification inside Apple they break the law. Not even the FBI are allowed to do that. Only NMCEC is an allowed recipient by US federal law.

Seems to be a misunderstanding between what the law appears to say and what the actual practice is. Law enforcement's interest is not served by trying to prosecute moderators or companies acting in good faith because they have CSAM in their possession.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#79

Post by foerbert »

shuckles wrote: post #36
mikepurvis wrote: post #22

Earlier quoted context omitted.

At least part of the concern is that a hash collision is basically "cause" for Apple to then dump and begin manually (like, with humans) reviewing the contents of your device, all of which will be happening behind the closed doors of a private corporation, outside of any of the usual oversight or innocent-presumption mechanisms that come from it happening through the courts. That, combined with a (pretty reasonable)…

That is literally the status quo with every cloud service. Apple, unlike the others, has said that they will evaluate you on the basis of what’s included in the associated data of your safety voucher, and you can inspect those contents because they’re shipped in the client. Facebook, for all I know, might be calculating a child predator likelihood score on my account based on how often I look up my middle school ex-g…

I don't think we can just appeal to the status quo here and assume it's acceptable. There's a couple reasons.

First, how many people really understood this previously? Did society at large actually knowingly accept the current state of things, or did it just happen without most people realizing it? Even here on HN where we'd expect to find people way more knowledgeable about it than in general I'm not sure how well known it was about what was actually happening, though I'd assume most would be aware it was possible.

Secondly, there's a significant difference between your own device or Apple's server doing this. On the technical side of things, right now, it might not matter that much since it currently is limited to things you upload to iCloud. But more philosophically, it's your own device being turned against you to check you for criminal behavior. That's very different from somebody else checking up on you after you willingly interact with them.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#80

Post by tester756 »

ulzeraj wrote: post #25
criticaltinker wrote: post #4

Earlier quoted context omitted.

The OP mentions that two countries have to agree to add a file to the list, but your concern is definitely valid: > Perhaps the most concerning part of the whole scheme is the database itself. Since the original images are (understandably) not available for inspection, it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights adv…

What if those two countries can be Poland and Hungary? These two countries have been passing lots of laws to ostracize and criminalize pro-LGBT content and are friendly to each other.

> These two countries have been passing lots of laws to criminalize pro-LGBT content

What do you mean

I live in one of those countries and I'd want to be aware

Post reply on HN