I was on the fence on this one because of Alex Stamos statements on this. I did expect to be able to trust him and in extension the Stanford Internet Observatory more.
Hash collision in Apple NeuralHash model
221–230 of 725 posts
Re: Hash collision in Apple NeuralHash model
#222Earlier quoted context omitted.
A police raid on a person's home, or even a gentler thorough search, can be enough to quite seriously disrupt a person's life. Certainly having the police walk away with all your electronics in evidence bags will complicate trying to work remotely. Of course, this is assuming everything works as intended and they don't find anything else they can use to charge you with something as they search your home. If you smoke…
This could happen with a perturbed image, but I doubt it. Apple will send the suspicious images to the relevant authorities. Those authorities will then look at the images. The chances are low that they will then seek a search, even though the images are innocent upon visual inspection. But maybe in some places a ping from Apple is good enough for a search and seizure.
In broad strokes, I agree with you. I think you're absolutely correct that most trained, educated, technologically sophisticated law enforcement bodies will do exactly as you suggest and decide that there's not enough to investigate.
That said, I'm not willing to say it won't happen. There are too many law enforcement entities of wildly varying levels of professionalism, staffing, and technical sophistication. Someone innocent, somewhere, is likely to have a multi-year legal drama because their local PD got an email from Apple.
And we haven't even gotten to subjects like how some LEOs will happily plant evidence once they decide you're guilty.
Re: Hash collision in Apple NeuralHash model
#223Earlier quoted context omitted.
Don't feel bad at all. I dumped macOS entirely from production workflow. I cannot work on computer knowing that something is "scanning" me and I am glad that my "paranoid" feeling stopped me to upgrade all office macs. Billionaires at (Apple) don't give a flying f*ck about users privacy. It is all vertical integration in the name of world domination. How removed from reality they are. This is week after Pegasus/NSO a…
> and there is no law who requires them to "scan" on device Yet . The demands by "concerned parents" (aka fronts for secret services, puritans/other religious fundamentalists and law-and-order hardliners) to "do something against child porn" have grown ever more strong and insane over the last years. (And you can bet that what is used on CSAM will immediately be used to go after legal pornography, sex work, drug enfo…
Why would they extend the CSAM scanner? It would be much simpler to just use all the OCR and image classification functions they have already deployed.
CSAM scanning is only useful for areas where Apple really doesn't want to even look at the actual material until they are extremely certain that it's a match. If they want to detect anti-government propaganda or something, there would be no such concerns, they would just do a regex search of your inbox. Infinitely more convenient.
Re: Hash collision in Apple NeuralHash model
#224Earlier quoted context omitted.
"Verified by a human" - and that human will be an overworked, underpaid, overseas subcontractor who may well have an incentive to mash the "Confirm match" button from time to time to improve his performance.
Beyond that, are we seriously going to ignore the fact that eventually this system will share the private photos of someone's naked kid with some random subcontractor? How is that even remotely OK? Or that it will find and share actual CSAM with said subcontractor?
This system does not use ML to find new CSAM images. It only checks for ones already in a known database. Your pictures of kids in the bathtub are not on the list.
What is show to the reviewer is a "visual derivative" which hasn't been clearly defined. A thumbnail image? Something with a censored section? We don't really know.
Re: Hash collision in Apple NeuralHash model
#225Earlier quoted context omitted.
For a normal random person, a grey man. In the real world, a single collision could be enough for the police to acquire further access to people they are already looking at. If the police want access to your phone for other reasons (drugs, taxes, illegal speech) they can use that one collision to get a warrant which will give them greater access. It is akin to cops wanting to search a car. They don't need a warrant.…
I'm not sure what you mean. If we are talking about Apple's new feature, they will not even be alerted until you have enough matches. And the police probably won't be using any neural hashes if they have access to your device, so I'm not sure what you are talking about.
That is an Apple current policy, not any sort of law. If the FBI wants to know if there are any collisions on a particular phone, that will be shared. If the FBI wants to know of all single collisions, that too must be shared. A corporation's internal policy decisions are nothing when faced with a government official carrying a warrant. If they have data indicating possible crimes, no matter how small, they can be forced to divulge it to investigators.
Unlike Lavabit, I don't think Apple will ever close up shop to protect customer data.
Re: Hash collision in Apple NeuralHash model
#226How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…
> 7. Apple reviewer confuses a featureless blob of gray with CSAM material, several times
I find it hard to believe that anyone has faith in any purported manual review by a modern tech giant. Assume the worst and you'll still probably not go far enough.
Re: Hash collision in Apple NeuralHash model
#227Earlier quoted context omitted.
Then, with all due respect, the attacker could just download actual CSAM. > If your adversary is the Mossad, YOU’RE GONNA DIE AND THERE’S NOTHING THAT YOU CAN DO ABOUT IT. The Mossad is not intimidated by the fact that you employ https:// . If the Mossad wants your data, they’re going to use a drone to replace your cellphone with a piece of uranium that’s shaped like a cellphone, and when you die of tumors filled wit…
Also, this XKCD: https://xkcd.com/538/ People are getting nerd-sniped about hash collisions. It's completely irrelevant. The real-world vector is that an attacker sends CSAM through one of the channels that will trigger a scan. Through iMessage, this should be possible in an unsolicited fashion (correct me if I'm wrong). Otherwise, it's possible through a hacked device. Of course there's plausible deniability here, b…
FWIW, this sort of argument may not trigger a change in policy, but a technical failure in the hashing algorithm might.
Re: Hash collision in Apple NeuralHash model
#228Earlier quoted context omitted.
They must be proven true beyond a reasonable doubt to get a person in a funny robe to put them in jail. Normal humans are not required to prove anything in order to think them.
> Normal humans are not required to prove anything in order to think them. No, but decent human beings are required to not think them true just because they're out there...
decent humans can have a lower standard for thinking someone did a crime than the criminal justice system does.
this is true whether you believe or disbelieve a rape claim, because if you disbelieve it you typically must believe the corrolary that the victim was committing the crime of lying to the police.
Re: Hash collision in Apple NeuralHash model
#229And everyone was saying how impossible this was. Not an expert, but I do enjoy how quickly this was absconded and disproven.
Impossible (or nearly so) for a cryptographically secure hash function, not a perceptual hash. When the stories on this first broke it wasn't entirely clear which type of hash Apple was using for their CSAM detection. (I assume now we know they're using NeuralHash?)
Re: Hash collision in Apple NeuralHash model
#230First CP. Then leaked or unauthorized nudes will get filtered. Filters for terrorists, and terrorist imagery or symbols. Start scanning for guns and drugs. Drug dealers and criminals get added to the list. How long before before it’s dissidents, political opponents, and minorities in dictatorships? How long before Tim Cooks CP filters are used against LGBT groups abroad?
It's funny how it's always CP or terrorism we need to watch out for.... Never is it the politician who accidentally went to war with wrong country, or the tax return of the congressman, or a cop sleeping with somebody who's under arrest, or EPA employee who took a bribe to declare a chemical safe, or mysteriously malfunctioning cameras in a prison the night of an alleged suicide.
Build surveillance to catch those people and I'll listen.