Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

631–640 of 725 posts

Re: Hash collision in Apple NeuralHash model

#631

Post by nullc »

hannasanarion wrote: post #543
nullc wrote: post #489

Earlier quoted context omitted.

The grey blob is a proof of concept. The existence of the original image is proof that not all images which produce the target hash are grey blobs. Since the grey blob exists, I believe it is fully possible to construct natural(-ish) images that have a selected hash. So, you should perhaps instead imagine attackers that modify lawful nude images to have matching hashes with child porn images. With that in mind, most…

The other thread demonstrates a similar attack with pictures of dogs. The same questions still apply. Why would you save tons of pictures of dogs to your photo library? Why would pictures of dogs look like child porn? Why would Apple reviewers think pictures of dogs are child porn? Why would the NCMEC think pictures of dogs are child porn? Why would law enforcement spend time arresting someone for pictures of dogs? W…

I'm really surprised to see a fellow HN poster making such a stark failure to generalize.

You see that they can do this with pictures of dogs. What makes you think they can't do exactly the same with pictures of crotches?

Presumably you don't believe there is some kind inherent dog-nature that makes dog images more likely to undermine the hash. :) People are using pictures of dogs because they are a tasteful safe-for-work example, not because anyone actually imagines using images of dogs.

An actual attack would use ordinary nude images, probably ones selected so that if you were primed to expect child porn you'd believe it if you didn't spend a while studying the image.

> where the same attack could not be performed more easily and causing more damage by actually using real CSAM images?

Actual child porn images are more likely to get deleted by the target and/or reported by the target. The attacker also takes on some additional risk that their possession of the images is a strict liability crime. This means that if the planting party gets found with the real child porn they'll be in trouble vs with the hash-matching-legal-images they'll only be in trouble if they get caught planting it (and potentially only exposed to a civil lawsuit, rather than a felony, depending on how they were planting it).

Personally, I agree that the second-preimage-images are not the most interesting attack! But they are a weakness that makes the system even more dangerous. We could debate how much more dangerous they make it.

Re: Hash collision in Apple NeuralHash model

#632

Post by ksec »

mda wrote: post #522
ksec wrote: post #362

Earlier quoted context omitted.

I actually want Apple to stand ground and implement this feature. Like you said the double down on PR and marketing was enough for me. I may not be dumping all iOS and Mac for now. But it was " the " definite signal and evidence this is no longer the old Steve Jobs's Apple. It is like watching Mark Zuckerberg talking about privacy when he doesn't understand anything about it. ( Or more like he has a different underst…

"Steve Jobs Apple"? I don't think Jobs would give a damn about people crying about Apple's decisions. I don't know why people thinks he would be a smidgen better than whoever managing apple after him.

Because he actually understand privacy better than 99.9% of people in Silicon Valley. He is also a product person who understand how users feel. Compare to current Apple which is "still" trying to give me a technical explanation of what is and what's not.

Re: Hash collision in Apple NeuralHash model

#633

Post by nullc »

nullc wrote: post #489
robertoandred wrote: post #379

Earlier quoted context omitted.

No, you couldn't. This is only checking images you upload to your iCloud photo library. Why would you save tons of gray blobs to your photo library? Why would gray blobs look like child porn? Why would Apple reviewers think gray blobs are child porn? Why would the NCMEC think gray blobs are child porn? Why would law enforcement spend time arresting someone for gray blobs?

The grey blob is a proof of concept. The existence of the original image is proof that not all images which produce the target hash are grey blobs. Since the grey blob exists, I believe it is fully possible to construct natural(-ish) images that have a selected hash. So, you should perhaps instead imagine attackers that modify lawful nude images to have matching hashes with child porn images. With that in mind, most…

Update: The latest results now have second-preimages that are other photos, not just noise. https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue... They still look pretty bad, but attacks only get better!

Re: Hash collision in Apple NeuralHash model

#634

Post by floatingatoll »

ksec wrote: post #626
floatingatoll wrote: post #405

Earlier quoted context omitted.

Steve Jobs would have implemented this in secret and never told us at all, same as they already did for iCloud photos so many years ago. That would have been a far better approach than today’s Apple is taking. Oh well.

Well Steve Jobs was the one that resisted PRISM and gave the middle finger to NSA.

NSA PRISM was and illegal and warrantless text analysis and search system for most communications on the Internet, collected all communications without bothering to filter at all, and had no protections in place to prevent people from randomly searching and reading content out of human curiosity.

Apple's CSAM implementation protects the user against algorithm defects, does not expose the user to legal trouble until they have at least 30 human-verified visual matches of CSAM content, and occurs on-device using a small set of confirmed and audited signatures to ensure that CSAM scanning requires a central system only for verifying the unverified, blurred, positive matches.

I would hesitate to compare Steve Jobs' views on PRISM to his likely views on something that is so clearly opposed to it in so many ways. So I do not yet understand your viewpoint that Apple CSAM scanning and PRISM would have been treated equally by Steve. Help me understand?

Re: Hash collision in Apple NeuralHash model

#635

Post by floatingatoll »

cyanite wrote: post #619
floatingatoll wrote: post #561

Earlier quoted context omitted.

China forced Apple by legislation to implement new iCloud algorithms for assigning China-region user data into China-hosted datacenters. Most countries, unlike the US, are not constrained by a requirement to only exercise previously-built mechanisms and not create new ones, in response to government demands. If China decides to require Apple to censor non-CSAM content on-device, they will do so whether or not CSAM co…

> China-hosted China-decryptable datacenters. China hosted, yes, but Apple denies China-decryptable, so that’s speculation unless you have a good source.

Nope, that's just me remembering wrong. Deleted those two words, thanks for the correction :)

Re: Hash collision in Apple NeuralHash model

#636

Post by cyanite »

sierpinsky wrote: post #529

"According to media reports, the cloud computing industry does not take full advantage of the existing CSAM screening toolsto detect images or videos in cloud computing storage. For instance, big industry players, such as Apple, do not scan their cloud storage. In 2019, Amazon provided only eight reports to the NCMEC, despite handling cloud storage services with millions of uploads and downloads every second. Others,…

> The whole discussion seems to center around what Apple intends to do on-device, ignoring what others are already doing in the cloud. Isn't this strange?

Very strange. Especially when this on-device technique means that Apple needs to access far less data than when doing it on the cloud.

Re: Hash collision in Apple NeuralHash model

#637

Post by __blockcipher__ »

falcolas wrote: post #347
topynate wrote: post #205

Earlier quoted context omitted.

This could happen with a perturbed image, but I doubt it. Apple will send the suspicious images to the relevant authorities. Those authorities will then look at the images. The chances are low that they will then seek a search, even though the images are innocent upon visual inspection. But maybe in some places a ping from Apple is good enough for a search and seizure.

FWIW, they won't send the images. Even in the pursuit of knocking back CSAM, there are strict restrictions on the transmission and viewing of CSAM - in some cases even the defendant's lawyers don't usually see the images themselves in preparation for a trial, just a description of the contents. Apple employees or contractors will likely not look at the images themselves, only visual hashes. They will instead contact…

> in some cases even the defendant's lawyers don't usually see the images themselves in preparation for a trial, just a description of the contents

Man that seems horrible. So you just have to trust the description is accurate? You’d think there’d at least be a “private viewing room” type thing (I get the obvious concern of not giving them a file to take home)

Re: Hash collision in Apple NeuralHash model

#638

Post by cyanite »

neximo64 wrote: post #29

Maybe the best idea is for a sufficient number of people to replicate the hashes with collisions from the CSAM database and make copies of photos with nothing in them like this one and just let Apple deal with it. Maybe it can have text too.

Best for whom? (Also, we don’t know the exact hash list Apple will use).

Re: Hash collision in Apple NeuralHash model

#639

Post by cyanite »

yayr wrote: post #112

How long did it take now to make the Apple algorithm ultimately useless or even harmful? Apple announcement of neural hashing: 5.8.2021. Generic algorithm to generate a different matching image: 8.8.2021. one script was already released 10 days ago here https://gist.github.com/unrealwill/c480371c3a4bf3abb29856c29...

None of this makes the system useless or harmful. Also, it’s not Apple’s algorithm. The actual hash list Apple will use is not accessible to the device.

Re: Hash collision in Apple NeuralHash model

#640

Post by neolog »

floatingatoll wrote: post #557
neolog wrote: post #523

Earlier quoted context omitted.

> CSAM scanning is irrelevant to 99.99999% of Apple's customers How long until an group of governments tells Apple to add Tank Man to the list?

Why would they bother? That's a terrible way to approach it. Just pass legislation requiring in-country datacenters that can be decrypted by thoughtcrime enforcers, like Russia and China are doing. Trying to get this done via a CSAM list that's absurdly closely audited would be a huge waste of time and not provide any significant benefit, and if such a request were ever made public, would likely result in severe poli…

Without the technology deployed, Apple can (and did) say they don't have the ability to break into users' phones.

If Apple deploys on-phone scanning, governments can just tell Apple to support a new list. It won't be the NCMEC CSAM list. It will be a "public safety and security" list. I wouldn't rule out underhandedness either. [1]

[1] https://www.nytimes.com/2020/07/01/technology/china-uighurs-...

Post reply on HN