Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ImageNet contains naturally occurring Apple NeuralHash collisions

blog.roboflow.com

Post reply on HN
Print view View original

431–440 of 530 posts

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#431

Post by heavyset_go »

mthoms wrote: post #349
rovr138 wrote: post #339

Earlier quoted context omitted.

On top of that, what happens if a court/government orders them to give them all the current data about people with matches, regardless of the 30 matches. They can't say if it is or not a match so they have to go after the individuals. Is that enough evidence for a warrant? Someone in the court thinks it's true and can't prosecute?, oh, it got leaked . -- Not every country has the same protections about innocent until…

As I understand it, Apple's servers know nothing until the 30+ match threshold is reached. This is actually one way that their system might be an improvement. NB: I'm not in favour of this system - I'm only commenting on this one specific scenario.

Yes, but the full pictures are uploaded to iCloud Photos for which Apple has the encryption keys, and can scan the photos whenever they please.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#432

Post by dkdk8283 »

t0mas88 wrote: post #337
dkdk8283 wrote: post #313

Earlier quoted context omitted.

This is already happening to phones. The baseband blobs are proprietary and most devices permit DMA. Nobody really knows what the blobs do. They likely have paved the way for Stingrays and other devices.

In Android yes, but Apple has an advantage here that they control the whole device. If Apple really wanted to they could secure the iPhone. Just offer a 2x higher bug bounty than any government and things like stingray would not work.

Apple is using Qualcomm modems today. Their own chip won’t be ready until 2023.

Although I think recent iphone designs don’t give the modem DMA so it’s still better than Android but I wouldn’t consider it secure.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#433

Post by stemlord »

incrudible wrote: post #395
csmpltn wrote: post #242

Earlier quoted context omitted.

> "The end result is that some peon at Apple has to look at the images and mark them as not CSAM. You've cost someone a bit of privacy, but that's it." This can be abused to spam Apple's manual review process, grinding it down to a halt. You've cost Apple time and money by making them review each such fake report.

> You've cost Apple time and money by making them review each such fake report. Ok, but… how do I profit? If I wanted to waste Apple employee time, I could surely find a way to do it, but why would I? The functioning of society relies on the fact that people generally have better things to do than waste each others time.

It's a form of protest, thus not seen as a waste of time by those doing it.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#434

Post by 8bitsrule »

int_19h wrote: post #346
rovr138 wrote: post #339

Earlier quoted context omitted.

On top of that, what happens if a court/government orders them to give them all the current data about people with matches, regardless of the 30 matches. They can't say if it is or not a match so they have to go after the individuals. Is that enough evidence for a warrant? Someone in the court thinks it's true and can't prosecute?, oh, it got leaked . -- Not every country has the same protections about innocent until…

More broadly speaking, every part of this scheme that is currently an arbitrary Apple decision (and not a technological limitation), can easily become an arbitrary government decision. And yes, it's true that the governments could always mandate such scanning before. The difference is that it'll be much harder politically for Apple to push back against tweaks to the scheme (such as lowering the bar for manual review…

Now that everyone can clearly see that 'Stallman was right', we're all just haggling about the price.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#435

Post by partdavid »

k2enemy wrote: post #419
vineyardmike wrote: post #264

Earlier quoted context omitted.

> The end result is that some peon at Apple has to look at the images and mark them as not CSAM. As others said, if the non-csam looks sexual at all, they'll probably get flagged for post-apple review. Beyond that, it doesn't seem to be in apple's interest to be conservative in flagging. An employee reviewer's best interest is to minimize false negatives not false positives. As many mentioned, even an investigation c…

> Beyond that, it doesn't seem to be in apple's interest to be conservative in flagging. An employee reviewer's best interest is to minimize false negatives not false positives. I would have thought the opposite. If there is a false positive that leads to an arrest and ruins someone's life, but the public sees that it is a false positive, then Apple will take an enormous hit in the marketplace. Nobody will want to ta…

Another commenter used the right phrase, "terribly asymmetry". There won't be any such thing in the public eye as a "false positive", only kiddie porn traders who got away with it.

> Nobody will want to take on the demonstrated real risk being falsely accused of possessing CSAM.

I don't think this is much of a risk. Defamation is difficult to prove, never applies to the law enforcement agencies who are going to make these arrests and "ruin people's lives", is never going to be down to Apple anyway (Apple is only referring things for investigation). I don't think you'll see much public indignance about this--even arguing against it in the real public eye (outside of wonky venues like HN) sounds tantamount to "supporting kiddie porn" in the naïve view of most members of the public.

> If they have a false negative, it is unclear to me what negative effects Apple would suffer. As far as I know, nobody would know about it outside of Apple.

This could potentially arise in any case with an abuser or producer or enabler or pimp or Maxwell/Epstein customer who has an iPhone; which is a lot of cases. As soon as Apple devices are supposed to "detect" kiddie porn, people will ask why people like this weren't caught earlier; and since Apple has money, they won't just ask this in the court of public opinion, they will sue for damages for abuses that "should have" been prevented by Apple's inspection of their pictures. Even if that's unlikely, it's much easier for a peon whose job it is to look at kiddie porn to just forward it on; and such a case really could damage Apple's optics.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#436

Post by mthoms »

heavyset_go wrote: post #431
mthoms wrote: post #349

Earlier quoted context omitted.

As I understand it, Apple's servers know nothing until the 30+ match threshold is reached. This is actually one way that their system might be an improvement. NB: I'm not in favour of this system - I'm only commenting on this one specific scenario.

Yes, but the full pictures are uploaded to iCloud Photos for which Apple has the encryption keys, and can scan the photos whenever they please.

Okay sure, but that's always been the case and has nothing to do with the system they've just introduced.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#438

Post by solidgol »

dang wrote: post #413
solidgol wrote: post #408

I think the headline is misleading. The point of the article is that the algorithm that Apple is using works as well as Apple says it does, and the headline implies that it doesn't.

What would be a better (i.e. more accurate and neutral) title?

A reply by the moderator! I am honored.

I think the headline should be:

ImageNet contains about as many naturally occurring NeuralHash collisions as Apple predicts

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#439

Post by judge2020 »

GatorD42 wrote: post #437

Wouldn’t the most likely attack be an angry (ex) spouse with physical access to a target’s phone placing images on it (and uploading to iCloud)? I haven’t seen this seemingly likely scenario discussed much.

two scenarios:

- Those images would either have to be actual CSAM (illegal to distribute in the first place and generally hard to get)

- fake collision images are placed instead, and all that would do is push the multiple fake CSAM images to Apple's human reviewers who would then likely mark it as a false positive/spam. If the Apple reviewer makes a mistake and does report it as CSAM them the only risk (in the US) is law enforcement having probably cause for search & seizure to look for other CSAM.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#440

Post by heavyset_go »

rootusrootus wrote: post #32
minitoar wrote: post #11

Earlier quoted context omitted.

The damage is already done by the time it gets to the point of devices being confiscated.

By the time the FBI comes knocking for your devices, they have a lot of evidence, not a list of hash collisions.

No, the FBI's job is to investigate whether or not a crime has been committed. They'll do this by getting a warrant for your devices, and then take them and scan them.
Post reply on HN