Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

431–440 of 725 posts

Re: Hash collision in Apple NeuralHash model

#431

Post by shapefrog »

falcolas wrote: post #366
shapefrog wrote: post #122

Earlier quoted context omitted.

I can guarantee nobody will see the inside of a courtroom, on charges of possession and distribution of child porn for possessing multiple images of grey noise (unless there is some steganography going on).

One does not need to go to court to have their life ruined by accusations. Ironically, there's quite a few examples of this over the years for alleged CSAM. One example that sticks out in my mind is a pair of grandparents who photographed their grandchildren playing in the back yard. A photo tech flagged their photo, they were arrested, and it took their lawyer going through the hoops to get a review of the photo for…

Sure, there are always cases - but was their photo of a grey blob that matched a hash but is clearly a grey blob or of a naked child?

If the photo was a grey blob and they had to go through a judicial review for someone to look at the photo and confirm 'yes that is a grey blob' then color me wrong.

Re: Hash collision in Apple NeuralHash model

#432

Post by teitoklien »

bengale wrote: post #419

I don't understand why this is a concern. Someone would need 30 or so child porn images to generate these. Then they'd need to create these grey blobs and send them to the target. The target would need to save them in their photo library for some reason so they sync to iCloud. Then when all this triggers the review they'll see they are grey blobs and nothing else will happen?

Its astonishing , How society went from keeping personal photos private , to thinking its ok , to let a random apple employee scan their photos for suspicion and debate on how human reviewers from a faceless corporation can easily stop these mistakes there.

I’d trust the algorithm more than the human reviewing it , considering the algorithm already shows flaws and loopholes , the human part of it does not bring any confidence.

This isnt facebook , this is your private photos , it shouldnt have reached till this stage , in the first place

Re: Hash collision in Apple NeuralHash model

#433

Post by coldtea »

Griffinsauce wrote: post #409
_jal wrote: post #153

Earlier quoted context omitted.

They must be proven true beyond a reasonable doubt to get a person in a funny robe to put them in jail. Normal humans are not required to prove anything in order to think them.

*in the US

I'm not in the US.

And even if it didn't hold in my country, it's still a tenet I think is fair.

So, I didn't intended invoke what's the legal requirements, but what should be the "right" thing (and what I think should also be legally mandated, even if it's not).

Re: Hash collision in Apple NeuralHash model

#434

Post by snowwrestler »

sandworm101 wrote: post #410
snowwrestler wrote: post #381

Earlier quoted context omitted.

Every AV (antivirus) software system scans customer-owned files on customer-owned devices. So the question is the same: if it’s easy for law enforcement to deputize such a system, where is the flood of cases built off of evidence gathered this way?

>> Every AV (antivirus) software system Mine doesn't. If Ubuntu or ClamAV is scanning all my photos and reporting the results to Canonical against my will then I will soon be having words with Mr. Linus.

If law enforcement can force software companies to adapt their software to serve law enforcement purposes, why aren’t Ubuntu or ClamAV doing so already? What makes them better at resisting requests from law enforcement than Apple?

Re: Hash collision in Apple NeuralHash model

#435

Post by labcomputer »

sitharus wrote: post #56
the8472 wrote: post #27

Earlier quoted context omitted.

> 7. Apple reviewer confuses a featureless blob of gray with CSAM material, several times A better collision won't be a grey blob, it'll take some photoshopped and downscaled picture of a kid and massage the least significant bits until it is a collision. https://openai.com/blog/adversarial-example-research/

So the person would have to accept and save an image that when looks enough like CSAM to confuse a reviewer…

So here's something I find interesting about this whole discussion: Everyone seems to assume the reviewers are honest actors.

It occurs to me that compromising an already-hired reviewer (either through blackmail or bribery) or even just planting your own insider on the review team might not be that difficult.

In fact, if your threat model includes nation-state adversaries, it seems crazy not to consider compromised reviewers. How hard would it really be for the CIA or NSA to get a few of their (under cover) people on the review team?

Re: Hash collision in Apple NeuralHash model

#436

Post by falcolas »

shapefrog wrote: post #431
falcolas wrote: post #366

Earlier quoted context omitted.

One does not need to go to court to have their life ruined by accusations. Ironically, there's quite a few examples of this over the years for alleged CSAM. One example that sticks out in my mind is a pair of grandparents who photographed their grandchildren playing in the back yard. A photo tech flagged their photo, they were arrested, and it took their lawyer going through the hoops to get a review of the photo for…

Sure, there are always cases - but was their photo of a grey blob that matched a hash but is clearly a grey blob or of a naked child? If the photo was a grey blob and they had to go through a judicial review for someone to look at the photo and confirm 'yes that is a grey blob' then color me wrong.

I'd view a "grey blob" to be the MVP of hash collisions. I doubt that this will end with grey blobs - I see it ending with common images (memes would be great for this) being invisibly altered to collide with a CSAM hash.

Re: Hash collision in Apple NeuralHash model

#437

Post by mmcwilliams »

Bjartr wrote: post #307
bostonsre wrote: post #259

Earlier quoted context omitted.

Why would someone do that? Why not just send the original if both are flagged as the original?

Because having actual CSAM images is illegal.

Doesn’t that make step 1 more dangerous for the attacker than the intended victim? And following this through to its logical conclusion; the intended victim would have images that upon manual review by law enforcement would be found to be not CSAM.

Re: Hash collision in Apple NeuralHash model

#438

Post by nojito »

notquitehuman wrote: post #282
nojito wrote: post #179

Any matches are matched again server side to thwart this type of attack. >Once Apple's iCloud Photos servers decrypt a set of positive match vouchers for an account that exceeded the match threshold, the visual derivatives of the positively matching images are referred for review by Apple. First, as an additional safeguard, the visual derivatives themselves are matched to the known CSAM database by a second, indep…

It doesn’t matter what they do after “looking for something to report to law enforcement.” Nothing after that makes it less invasive.

Disagree. What companies do now is much more invasive. (Indiscriminately scanning cloud storage)

Re: Hash collision in Apple NeuralHash model

#439

Post by dylan604 »

jtmarl1n wrote: post #352
dylan604 wrote: post #232

Earlier quoted context omitted.

I've only seen Apple admit defeat once, and that was regarding the trashcan MacPro. Otherwise, it's "you're holding it wrong" type of victim blaming as they quietly revise the issue on the next version. Can anyone else think of times where Apple has admitted to something bad on their end and then reversed/walked away from whatever it was?

The Apple AirPower mat comes to mind although there are rumors they haven't abandoned the effort completely. Butterfly keyboard seems to finally be acknowledged as a bad idea and took several years to get there. The next Macbook refresh will be interesting as there are rumors they are bring back several I/O ports that were removed when switching to all USB-C. I agree with your overall point, just some things that cam…

ah yes, the butterfly keyboard. i must have blocked that from my mind after the horror it was. although, they didn't admit anything on that one. that was just another "you're holding it wrong" silent revision that was then touted as a new feature (rather than oops we fucked up).

The trashcan MacPro is still the only mea culpa I am aware of them actually owning the mistake.

The Airpower whatever was never really released as a product though, so it is a strange category. New question, is the Airpower whatever the only product offically announced on the big stage to never be released?

Re: Hash collision in Apple NeuralHash model

#440

Post by labcomputer »

FabHK wrote: post #61
michaelt wrote: post #50

Earlier quoted context omitted.

Just yesterday, here on HN there was an article [1] about adversarial attacks that could make road signs get misread by ML recognition systems I'd be astonished if it wasn't possible to do the same thing here. [1] https://news.ycombinator.com/item?id=28204077

But the remarkable thing there (and with all other adversarial attacks I've seen) is that the ML classifier is fooled, while for us humans it is obvious that it is still the original image (if maybe slightly perturbed). But in the case of Apple's CSAM detection, the collision would first have to fool the victim into seeing an innocent picture and storing it (presumably, they would not accept and store actual CSAM [^]…

> then fool the human reviewer into also seeing CSAM (unlike the innocent victim).

Or just blackmail and/or bribe the reviewers. Presumably you could add some sort of 'watermark' that would be obvious to compromised reviewers. "There's $1000 in it for you if you click 'yes' any time you see this watermark. Be a shame if something happened to your mum."

Post reply on HN