Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

511–520 of 725 posts

Re: Hash collision in Apple NeuralHash model

#511

Post by shapefrog »

falcolas wrote: post #484
shapefrog wrote: post #441

Earlier quoted context omitted.

If you need a judicial review to confim that a slightly altered Bernie in Coat and Gloves meme is not the same image as the picture of a child being raped that they have on file then we have way bigger problems.

Here's the thing with CSAM - it's illegal to view and transmit. So nobody, until the police have confiscated your devices, will actually be able to verify that it is a "child being raped." They'll view visual hashes, look at descriptions, and so forth, but nobody from Apple will actually be looking at them, because then they are guilty of viewing and transmitting CSAM. I noted in another comment, even the prosecutors…

This is just not true. Human review is conducted. Apple will conduct human review, facebook conduct human review, NCMEC will conduct human review, law enforcement will conduct human review, lawyers and judges will conduct human review.

Over the years there have been countless articles etc about how fucked up being a reviewer of content flagged at all the tech companies is. https://www.vice.com/en/article/a35xk5/facebook-moderators-a...

Re: Hash collision in Apple NeuralHash model

#512

Post by floatingatoll »

dmix wrote: post #473
floatingatoll wrote: post #405

Earlier quoted context omitted.

Steve Jobs would have implemented this in secret and never told us at all, same as they already did for iCloud photos so many years ago. That would have been a far better approach than today’s Apple is taking. Oh well.

There’s a significant leap from implementing something server side to on the consumer handheld devices themselves. Even if it’s just similar software it is regardless much more serious. I personally thought the unencrypted backups was enough of a death-knell as it provides everything on your phone but anything with real-time on device access is always a gold mine for surveillance hawks.

It's serious to you, but CSAM scanning is irrelevant to 99.99999% of Apple's customers, and Jobs would never have allowed an announcement about migrating CSAM scanning from uploads to the cloud to the device uploading to the cloud. That's an implementation detail that wouldn't be relevant to discuss with outsiders. Instead, I expect he would have presented it in a closed session to the FBI and/or Congress. Never to the press, not like this.

Re: Hash collision in Apple NeuralHash model

#513

Post by endisneigh »

MisterSandman wrote: post #368
endisneigh wrote: post #308

Earlier quoted context omitted.

If it doesn’t look like it wouldn’t a human reviewer disregard it once it gets to that point? Personally I don’t really see the issue.

We don't know how the human review is going to work. Countries with fewer resources are going to find it easier to just arrest/detain any suspects, instead of spending time and money on figuring out which reports are true. All you have to be is accused of CP for your life to be destroyed. It doesn't matter if you did it or not.

What you’re describing is possible even if you didn’t receive anything.

If the government wants to get you they don’t need this Apple scanning tech, or anything at all really

Re: Hash collision in Apple NeuralHash model

#514

Post by tandav »

dmix wrote: post #473
floatingatoll wrote: post #405

Earlier quoted context omitted.

Steve Jobs would have implemented this in secret and never told us at all, same as they already did for iCloud photos so many years ago. That would have been a far better approach than today’s Apple is taking. Oh well.

There’s a significant leap from implementing something server side to on the consumer handheld devices themselves. Even if it’s just similar software it is regardless much more serious. I personally thought the unencrypted backups was enough of a death-knell as it provides everything on your phone but anything with real-time on device access is always a gold mine for surveillance hawks.

until ios source code is closed all privacy claims is only backed by trust. They easily can do whatever they want if you're not compiling from source. There's no way to ensure your data is not leaving your iphone/mac with some "system" network requests.

Re: Hash collision in Apple NeuralHash model

#515

Post by shapefrog »

dannyw wrote: post #492
shapefrog wrote: post #468

Earlier quoted context omitted.

The process would not trigger any action. The NCMEC, who can look at the material, and are the people to whom the matter is reported, would compare the flagged image with the source material and reject it as not matching known CSAM. What if the legal porn of a 21 year old that triggered the collision match looked really really really close? So close that a human can not distinguish between the image of a 12 year old…

You are aware that a lot of CSAM are close ups of say pussies for example, and human anatomy can look very similar? I'm not talking about images of rape here. I'm taking about images that you'd see on a regular porn site, of adults and their body parts. You are also aware that CSAM covers anywhere from 0 to 17.99 years of age, and the legal obligation to report exists equally for the whole spectrum? So let's say I do…

If you have content that has a matching hash value and is identical by all computational and human inspection to content that has been identified as CSAM from which the hash was generate then you have a problem.

Without getting into the metaphysics of what is an image, at that point, you basically have a large collection of child porn.

Your hypothetical oppressive regime has gone to a lot of trouble planting not illegal evidence on your device. It would be much more effective to just put actual child porn on your device, which you would need to have to conduct the attack in the first place.

Re: Hash collision in Apple NeuralHash model

#516

Post by floatingatoll »

dannyw wrote: post #478
floatingatoll wrote: post #336

Earlier quoted context omitted.

State actors will just Gitmo you, without all this wasteful effort on hashes. This system offers no benefit sufficient to make it worth their time if they want to cull you from the population somehow.

No, China can't just Gitmo an American journalist on American soil. But now China can send some legal pornography (eg closeup pussy pictures), disturbed to match a CSAM hit, to a journalist they don't like and get them in jail. Why can't China do this before? Because previously, they'd still need to tip off authorities, which has an attribution trail and credibility barrier. Now, they can just use Pegasus to plant th…

The scenario you describe has already been extant for the past ten years. Unreported zerodays could have been used at any time to inject a CSAM hit into someone's camera roll, way back in time where they wouldn't see it, in order to get them investigated. Their phone would have uploaded it to iCloud or Google Photos or Dropbox Whatever and the CSAM detections at each place would have fired off. No need for any of this fancy AI static nonsense.

I know of zero instances of this attack being executed on anyone, so apparently even though it's been possible for years, it isn't a material threat to any Apple customers today. If you have information to the contrary, please present it.

What new attacks are possible upon device owners when the CSAM scanning of iCloud uploads is shifted to the device, that were not already a viable attack at any time in the past decade?

Re: Hash collision in Apple NeuralHash model

#517

Post by bengale »

teitoklien wrote: post #505
bengale wrote: post #499

Earlier quoted context omitted.

It's not an example though. Putting aside a lack of news report we'll assume it's true, it's got nothing to do with cloud scanning for images. A picture of someones children wouldn't trigger these systems at all.

> its got nothing to do with cloud scanning for images Everyone else are cloud services scanning for images , cloud or on-device wont change the math, math stays same and here the math makes mistake > A picture of someone’s children wouldn’t trigger these systems at all The main post under which we are conversing , is about someone having generated a collision to trigger this system.

I feel we're circling the same issues here so I'll leave it after this. But what you're saying is the point I'm making, many clouds have been doing this for a decade and we've not seen the issues you're worried about. The one example you gave was a photo tech, so human developing photos, making a mistake. Not someone being targeted with material that would trigger google photos or Facebook to flag them, not algorithms making mistakes with photos of someones kids, none of it.

The post we're commenting under has created hash collisions with grey blob images, if you get 100 of these they won't get past the review step anyway. It'd be a pointless attack.

Like most of the outrage around this system it mostly seems to boil down to FUD.

Re: Hash collision in Apple NeuralHash model

#518

Post by russdpale »

fsloth wrote: post #70
FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

"How can you use it for targeted attacks?" Just insert a known CSAM image on target's device. Done. I presume this could be used against a rival political party to ruin their reputation - insert bunch of CSAM images on their devices. "Party X is revealed as an abuse ring". This goes oh-so-very-nicely with Qanon conspiracy theories which even don't require any evidence to propagate widely. Wait for Apple to find the i…

You don't have to add a real picture, just add the hash of a benign, (new) cat picture to the db, then put the cat picture on the phone, then release a statement saying the person has popped up on your list. By the time the truth comes out the damage is done.

Re: Hash collision in Apple NeuralHash model

#519

Post by drvdevd »

ByteWelder wrote: post #247
vegetablepotpie wrote: post #197

Earlier quoted context omitted.

> I’ve dumped the entire iOS ecosystem in the last week. Not to go off topic from your main point, but what did you move to?

I'm not the one you responded to, but did something similar and feel like sharing: - Desktop: Manjaro Gnome, for that amazing macOS-like desktop. It even does the 3 finger swipe up to see all your apps with Apple's Touchpad. - Phone: OnePlus 8T with microG variant of LineageOS (alternatives: Pixel line-up), because that allows me to still receive push notifications. I have over 40 apps and only found 1 that didn't wo…

Thanks for sharing. Time to start compiling these lists.
Post reply on HN