Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

191–200 of 725 posts

Re: Hash collision in Apple NeuralHash model

#191

Post by tzs »

user-the-name wrote: post #25
csande17 wrote: post #17

Earlier quoted context omitted.

> It also requires getting a hold of actual blacklisted hashes, which I doubt anyone has, unless they have actual child pornography. I've never personally seen or looked for any images like this, but if they weren't already proliferating online and widely available to criminals, why would we need to build an elaborate client-side scanning system to detect and report people who have copies of them?

Do you have access to darknet child pornography trading networks? I don't.

OK, now I'm curious. How does darknet stuff actually work?

Is there some darknet search engine that you access over Tor and type whatever illegal thing you seek such as "child porn" or "stolen credit cards" or "heroin" or "money laundering" into and it points you to providers of those illegal goods and services reachable over some reasonably secure channel?

Or is it one of those things where someone already using a particular child porn or stolen card or money laundering or heroin selling site has to put you in contact with them?

Re: Hash collision in Apple NeuralHash model

#192

Post by ta988 »

avianlyric wrote: post #119
FabHK wrote: post #111

Earlier quoted context omitted.

Depends very much on the process Apple uses to make the "visual derivative", though. Also, defence by producing the original innocuous image (and showing that it triggers both parts of Apple's process, NeuralHash and human review of the visual derivative) should be possible, though a lot of damage might've been done by then.

> Also, defence by producing the original innocuous image At this point you’re already inside the guts of the justice system, and have been accused of distributing CSAM. Indeed depending on how diligent the prosecutor is, you might need to wait till trial before you can defend yourself. At that point you’re life as you know is already fucked. The only thing proving your innocence (and the need to do so is itself a co…

And now you will be accused of trying to hide illegal material in innocuous images.

Re: Hash collision in Apple NeuralHash model

#193

Post by kleene_op »

sitharus wrote: post #56
the8472 wrote: post #27

Earlier quoted context omitted.

> 7. Apple reviewer confuses a featureless blob of gray with CSAM material, several times A better collision won't be a grey blob, it'll take some photoshopped and downscaled picture of a kid and massage the least significant bits until it is a collision. https://openai.com/blog/adversarial-example-research/

So the person would have to accept and save an image that when looks enough like CSAM to confuse a reviewer…

Not necessarily.

If you know the method used by Apple to scale down flagged images before they are sent for review, you can make it so the scaled down version of the image shows a different, potentially misleading one instead:

https://thume.ca/projects/2012/11/14/magic-png-files/

At the end of the day:

- You can trick the user into saving an innocent looking image

- You can trick Apple NN hashing function with a purposely generated hash

- You can trick the reviewer with an explicit thumbnail

There is no limit to how devilish one can be.

Re: Hash collision in Apple NeuralHash model

#194

Post by Kalium »

topynate wrote: post #185

Second preimage attacks are trivial because of how the algorithm works. The image goes through a neural network (one to which everyone has access), the output vector is put through a linear transformation, and that vector is binarized, then cryptographically hashed. It's trivial to perturb any image you might wish so as to be close to the original output vector. This will result in it having the same binarization, he…

A police raid on a person's home, or even a gentler thorough search, can be enough to quite seriously disrupt a person's life. Certainly having the police walk away with all your electronics in evidence bags will complicate trying to work remotely.

Of course, this is assuming everything works as intended and they don't find anything else they can use to charge you with something as they search your home. If you smoke cannabis while being in the wrong state, you're now in several more kinds of trouble.

Re: Hash collision in Apple NeuralHash model

#195

Post by DebtDeflation »

sneak wrote: post #52
brigandish wrote: post #49

Earlier quoted context omitted.

I think you may have attracted less downvotes if the phrasing was changed to "Yes, just like false accusations of rape , it doesn't matter that you prove it was false afterwards." I also think that those downvoting you might've applied the principle of charity and taken the best interpretation of what you've written or at least ask .

All criminal accusations, including true ones , should be treated as false until the accused is proven guilty. This is a fundamental tenet of human rights in western, small-l liberal free societies. The fact that this is controversial these days is literally insane to me. The consequences of throwing this fundamental system out the window is that you get the sort of nonsense that happened with Assange, where he was l…

The real problem occurs long before a verdict is rendered in a court of law. Someone gets arrested for a misdemeanor, they spend a couple of nights in jail until bail can be arranged, in the meantime they got fired for missing work, the car they were driving was towed/impounded and will cost them hundreds of dollars to retrieve, they missed their rent payment and their landlord has begun eviction proceedings, etc. The fact that they are found not guilty when their trial happens a year later is irrelevant. Not to mention every future potential employer Googles their name and the first link that comes up is their arrest record. The "presumption of innocence" is meaningless when so much damage is done long before a trial even starts.

Re: Hash collision in Apple NeuralHash model

#196

Post by bitneuker »

jefftk wrote: post #138
reacharavindh wrote: post #132

I admit that I have not done enough research to have a strong opinion on this, but why is Apple taking this on themselves? As far as I can see, this outrage is because of "scanning on iPhone" that is wildly out of user's control. Why can't Apple be like others and say we scan the shit out of what you upload to iCloud(and it is in our Terms and Conditions to use iCloud)? Almost all tech people know that iCloud (or its…

The speculation that I've seen here is that Apple is rolling this out ahead of a future announcement that iCloud uploads will be encrypted.

Are they not? I would've thought that with Apple advertising privacy and security they would at least encrypt iCloud uploads.

Re: Hash collision in Apple NeuralHash model

#197

Post by vegetablepotpie »

hughrr wrote: post #7

That’s end game. Now you can use it for targeted attacks against innocent people. This needs to be shut down and disposed of immediately. There is no other outcome which is socially acceptable for Apple. I feel vindicated now. There are a lot of people saying that I’m insane as I’ve dumped the entire iOS ecosystem in the last week. But Craig was busy steamrolling out the marketing still only a couple of days back abo…

> I’ve dumped the entire iOS ecosystem in the last week.

Not to go off topic from your main point, but what did you move to?

Re: Hash collision in Apple NeuralHash model

#199

Post by hughrr »

vegetablepotpie wrote: post #197
hughrr wrote: post #7

That’s end game. Now you can use it for targeted attacks against innocent people. This needs to be shut down and disposed of immediately. There is no other outcome which is socially acceptable for Apple. I feel vindicated now. There are a lot of people saying that I’m insane as I’ve dumped the entire iOS ecosystem in the last week. But Craig was busy steamrolling out the marketing still only a couple of days back abo…

> I’ve dumped the entire iOS ecosystem in the last week. Not to go off topic from your main point, but what did you move to?

Linux, dumbphone, DSLR. This was the final straw on a planned exit to be honest.

Re: Hash collision in Apple NeuralHash model

#200

Post by Grustaf »

nbzso wrote: post #106
hughrr wrote: post #7

That’s end game. Now you can use it for targeted attacks against innocent people. This needs to be shut down and disposed of immediately. There is no other outcome which is socially acceptable for Apple. I feel vindicated now. There are a lot of people saying that I’m insane as I’ve dumped the entire iOS ecosystem in the last week. But Craig was busy steamrolling out the marketing still only a couple of days back abo…

Don't feel bad at all. I dumped macOS entirely from production workflow. I cannot work on computer knowing that something is "scanning" me and I am glad that my "paranoid" feeling stopped me to upgrade all office macs. Billionaires at (Apple) don't give a flying f*ck about users privacy. It is all vertical integration in the name of world domination. How removed from reality they are. This is week after Pegasus/NSO a…

How likely is it that you will have enough colliding images in your photo library to even trigger a review?

I'm guessing you need at least 5 images, perhaps much more, to trigger it. In any case, 1 image is definitely not enough.

Post reply on HN