Can someone help me figure out why this is such a big deal? Thanks
Hash collision in Apple NeuralHash model
411–420 of 725 posts
Re: Hash collision in Apple NeuralHash model
#412Earlier quoted context omitted.
This system does not use ML to find new CSAM images. It only checks for ones already in a known database. Your pictures of kids in the bathtub are not on the list. What is show to the reviewer is a "visual derivative" which hasn't been clearly defined. A thumbnail image? Something with a censored section? We don't really know.
Yes I'm aware that it checks against a known database but clearly there can be collisions. So eventually it will share someone's private images.
It would need to have 30 collisions before anything even took place, which realistically isn't going to happen.
Re: Hash collision in Apple NeuralHash model
#413Earlier quoted context omitted.
One does not need to reverse the CSAM hashes to find a collision with a hash. If the evaluation is being done on the phone, including identifying a hash match, the hashes must also be on the phone.
No, matches are not verified on the phone. On the phone, your image hash is used to look up an encrypted/blinded (via the server's secret key) CSAM hash. Then your image data (the hash and visual derivative) is encrypted with that encrypted/blinded hash. This encrypted payload, along with a part of your image's hash, is sent to Apple. Then on the server, Apple uses that part of your image's hash and their secret key…
To quote a sibling comment, who looked into the horses' mouth:
> Apple’s method of detecting known CSAM is designed with user privacy in mind. Instead of scanning images in the cloud, the system performs on-device matching using a database of known CSAM image hashes provided by NCMEC and other child-safety organizations. Apple further transforms this database into an unreadable set of hashes, which is securely stored on users’ devices.
https://www.apple.com/child-safety/pdf/CSAM_Detection_Techni...
Re: Hash collision in Apple NeuralHash model
#414Earlier quoted context omitted.
FWIW, they won't send the images. Even in the pursuit of knocking back CSAM, there are strict restrictions on the transmission and viewing of CSAM - in some cases even the defendant's lawyers don't usually see the images themselves in preparation for a trial, just a description of the contents. Apple employees or contractors will likely not look at the images themselves, only visual hashes. They will instead contact…
On reflection, yes, there must be warrants involved. I'm raising my estimate of how likely it is that innocent people get raided due to this. The warrant would properly only be to search iCloud, not some guy's house, but I can easily see overly-broad warrants being issued.
> The warrant would properly only be to search iCloud,
iCloud is encrypted, so that warrant is useless.
They need to unlock and search the device.
Re: Hash collision in Apple NeuralHash model
#415Earlier quoted context omitted.
> " Which Apple will dutifully install and run, because they're required by local laws. " Which Apple have stated that they won't do, and have designed the system so they can't do that without it being found out: https://news.ycombinator.com/item?id=28221082 Can't you at least post accurate information about this system and support outrage based on facts instead of fantasy?
> Which Apple have stated that they won't do For your random off-of-the-mill dictatorship, yes. For the US? EU? China? India? No way they can refuse such a request from these markets. And if they could and get away with it, it would be a very worrying situation in itself regarding (democratic) control of government over global mega corporations.
If you believe "the government can compel any company to do anything", then this system is no worse than any other.
Although how do you think Linus Torvalds "manged to get away with refusing" adding backdoors? https://www.techdirt.com/articles/20130919/07485524578/linus...
Re: Hash collision in Apple NeuralHash model
#416Earlier quoted context omitted.
In China, iCloud is run by the government.
It's actually not, but even if it were, that would be yet another reason CSAM scanning is completely irrelevant for government spying. Any spying really. It's much easier to just look at the images themselves.
> It's actually not
It's Joe Wong's joke, that it's like peeing in the snow in a dark winter night, while there is a difference but it's really hard to tell.
Re: Hash collision in Apple NeuralHash model
#417Earlier quoted context omitted.
That's an incomplete statement. Currently, they must comply with warranty requests by scanning if they have the ability to scan . If they have no such ability (say, because they designed their phones from a privacy-first perspective), the law makes no requirement that they create such a capability. And that's what pisses people off about this.
Warrants are only part of the issue. The bigger issue is civil and criminal liability. If someone uploads child porn to iCloud and then shares it with someone else Apple is possessing and distributing child porn. Today they aren't liable because there is a law that shields them. But that law comes with strings attached around assisting law enforcement. By doing an end run around those strings Apple is not holding up…
> If someone uploads child porn to iCloud and then shares it with someone else Apple is possessing and distributing child porn.
If someone sends CSAM using Federal Express, is FedEx legally liable for "possessing and distributing" that material? Does FedEx need to start opening up packages and scanning hard drives, DVDs, USB sticks, etc. to ensure that they don't contain any CSAM or other illegal data?
I really struggle with the lengths that people are going to to justify these moves. If we can justify this, it's pretty simple to justify a lot more surveillance as well. CSAM is not the only scourge in our society.
Reminder: Apple tells us that they consider privacy a "fundamental human right"[1]. That simply does not square with their recent announcement of on-device scanning, and some would argue that it does not square with scanning or content analysis anywhere, especially on behalf of government.
Re: Hash collision in Apple NeuralHash model
#418First CP. Then leaked or unauthorized nudes will get filtered. Filters for terrorists, and terrorist imagery or symbols. Start scanning for guns and drugs. Drug dealers and criminals get added to the list. How long before before it’s dissidents, political opponents, and minorities in dictatorships? How long before Tim Cooks CP filters are used against LGBT groups abroad?
This is the real danger here. Covid made them bold, it set them up as a judge of content passing through their systems. Now they are consolidating themselves in that role. A boot stamping on a human face for all eternity.
Fact checking to censoring misinformation then on device content scanning. In less than 5 years we have made multiple leaps towards censorship.
Re: Hash collision in Apple NeuralHash model
#419I don't understand why this is a concern. Someone would need 30 or so child porn images to generate these. Then they'd need to create these grey blobs and send them to the target. The target would need to save them in their photo library for some reason so they sync to iCloud. Then when all this triggers the review they'll see they are grey blobs and nothing else will happen?
Re: Hash collision in Apple NeuralHash model
#420How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…
Actually, need to run step 1-3 at least 30 times
You can do steps 2-3 all in one step "Hey Bob, here's a zip file of those funny cat pictures I was telling you about. Some of the files got corrupted and are grayed out for some reason".