Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

371–380 of 725 posts

Re: Hash collision in Apple NeuralHash model

#371
post #7

That’s end game. Now you can use it for targeted attacks against innocent people. This needs to be shut down and disposed of immediately. There is no other outcome which is socially acceptable for Apple. I feel vindicated now. There are a lot of people saying that I’m insane as I’ve dumped the entire iOS ecosystem in the last week. But Craig was busy steamrolling out the marketing still only a couple of days back abo…

You feel vindicated because someone made one image look like another?

Re: Hash collision in Apple NeuralHash model

#372
Some people here in comments believe that whoever gonna check reported material on Apple side will never ever flag false-positive. We already know that NCMEC database itself don't exclusively contain child porn, but also some other photos that closely related ot CSAM. Even if those photos don't have actual CSAM on them. But let's ignore this fact.

Do people who believe in behevolent Apple understand that CSAM don't all have some big red "CHILD PORN" sign on it? No demonic feel included. Like any porn we can suppose that many of such images might not even have any faces or literally anything that make them different from images of 18yo person.

When you think well about it brute-forcing actual porn or some jailbait images doesn't sound that impossible. All you need is a lot of totally legal porn and some compute power. Both we have in abundance.

Re: Hash collision in Apple NeuralHash model

#373

Earlier quoted context omitted.

I find it very hard to believe that Apple will put literal future of iPhone (just imagine the amount of bad press if one false negative comes out of review process and is reported to the authorities) to "overworked, underpaid overseas subcontractor". Apple is greedy, ruthless, tone-deaf machine, but I don't think they're stupid.

That’s already how app reviews work.

I think rejecting one of million apps and angering one developer is a bit different than reporting someone about a child pornography/abuse, ruining their life and making a global scandal that can put future iPhone sales in jeopardy.

Re: Hash collision in Apple NeuralHash model

#374
post #350

Earlier quoted context omitted.

Can a warrant compel them to develop the capability?

The law gets debatable here. The warrant can ultimately be served only to the owner/responsible party for the system. If apple decides that they own all iPhones, a claim they've loosely made for a long time re Flash etc. Then they could be served a warrant on the device. If they just sold the device to someone, then the police would have to issue a warrant to that individual. My understanding is that as an individual…

Apple does not claim they own all iPhones.

The legal principle is the same, actually. The law says you can install whatever software you want on an iPhone and Apple cannot stop you (jailbreaking is legal). But the law cannot force Apple to make it easy for you to do so.

Re: Hash collision in Apple NeuralHash model

#375
post #248

Earlier quoted context omitted.

That's an incomplete statement. Currently, they must comply with warranty requests by scanning if they have the ability to scan . If they have no such ability (say, because they designed their phones from a privacy-first perspective), the law makes no requirement that they create such a capability. And that's what pisses people off about this.

If Apple launches a system for comparing iCloud uploads to a third-party hash list, then adding the ability to do targeted scans for arbitrary additional law-enforcement-provided hashes would also be a form of creating capability. The people getting pissed off about this have not, so far as I’ve seen, demonstrated why the law would require Apple to add the capability for targeted scans of arbitrary hashes. Police can…

>> They can’t use a warrant to force you to videotape someone

Maybe in the narrow context of US domestic child pornography investigations. In the wider world it is very possible for police to force such things. Even in the US, CALEA demands that certain companies develop abilities that they would not normally want (interception). The principal that US companies need not actively participate in police investigations disappeared decades ago. On the international level, all bets are off.

https://en.wikipedia.org/wiki/Communications_Assistance_for_...

"by requiring that telecommunications carriers and manufacturers of telecommunications equipment modify and design their equipment, facilities, and services to ensure that they have built-in capabilities for targeted surveillance [...] USA telecommunications providers must install new hardware or software, as well as modify old equipment, so that it doesn't interfere with the ability of a law enforcement agency (LEA) to perform real-time surveillance of any telephone or Internet traffic."

Re: Hash collision in Apple NeuralHash model

#376

Earlier quoted context omitted.

This could happen with a perturbed image, but I doubt it. Apple will send the suspicious images to the relevant authorities. Those authorities will then look at the images. The chances are low that they will then seek a search, even though the images are innocent upon visual inspection. But maybe in some places a ping from Apple is good enough for a search and seizure.

FWIW, they won't send the images. Even in the pursuit of knocking back CSAM, there are strict restrictions on the transmission and viewing of CSAM - in some cases even the defendant's lawyers don't usually see the images themselves in preparation for a trial, just a description of the contents. Apple employees or contractors will likely not look at the images themselves, only visual hashes. They will instead contact…

On reflection, yes, there must be warrants involved. I'm raising my estimate of how likely it is that innocent people get raided due to this. The warrant would properly only be to search iCloud, not some guy's house, but I can easily see overly-broad warrants being issued.

Re: Hash collision in Apple NeuralHash model

#377
post #87

Earlier quoted context omitted.

I'm sure the reviewers will definitely be able to give each reported image enough time and attention they need, much like the people youtube employs to review videos discussing and exposing animal abuse, holocaust denial and other controversial topics.

Difference in volume. Images that trip CSAM hash are a lot rarer than the content you just described.

I personally am not aware of how the perceptual hash values are distributed in it's keyspace. Perceptual hashes can have issues with uniform distribution, since they are somewhat semantic and most content out there is too. As such, I wouldn't make statements about how often collisions would occur.

Re: Hash collision in Apple NeuralHash model

#378

Earlier quoted context omitted.

You cannot extract or reverse the CSAM hashes. They've been encrypted and blinded using server-side-only keys. If TFA said that, it's lying.

One does not need to reverse the CSAM hashes to find a collision with a hash. If the evaluation is being done on the phone, including identifying a hash match, the hashes must also be on the phone.

I believe the hash comparisons are made on Apple's end. Then the only way to get hashes will be a data breach on Apple's end (unlikely but not impossible) or generating it from known CSAM material.

Re: Hash collision in Apple NeuralHash model

#379
post #143
post #3

Expectation : Political rivals and enemies of powerful people will be taken out because c-ild pornography will be found in their phone. Pegasus can already monitor and exfiltrate every ounce of data right now, it won't be that hard to insert compromising images on the infected device. Any news about "c-ild porn" being found on someone's phone is suspect now. This has been done before : 1) https://www.deccanchronicle.…

That image planting virus concept is the scariest thing. Could you trigger this system and get the police alerted with spam mms with images or spam email with images, or targeted ads that get the gray blob images into your tmp files? Or does it have to be an actual script downloading images, say, one per week in secret until it triggers?

No, you couldn't. This is only checking images you upload to your iCloud photo library.

Why would you save tons of gray blobs to your photo library? Why would gray blobs look like child porn? Why would Apple reviewers think gray blobs are child porn? Why would the NCMEC think gray blobs are child porn? Why would law enforcement spend time arresting someone for gray blobs?

Re: Hash collision in Apple NeuralHash model

#380

Earlier quoted context omitted.

One does not need to reverse the CSAM hashes to find a collision with a hash. If the evaluation is being done on the phone, including identifying a hash match, the hashes must also be on the phone.

I believe the hash comparisons are made on Apple's end. Then the only way to get hashes will be a data breach on Apple's end (unlikely but not impossible) or generating it from known CSAM material.

That's not what Apple's plans state. The comparisons are done on phone, and are only escalated to Apple if there are more than N hash matches, at which point they are supposedly reviewed by Apple employees/contractors.

Otherwise, they'd just keep doing it on the material that's actually uploaded.

Post reply on HN