Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

201–210 of 725 posts

Re: Hash collision in Apple NeuralHash model

#201

Post by sandworm101 »

nbzso wrote: post #106
hughrr wrote: post #7

That’s end game. Now you can use it for targeted attacks against innocent people. This needs to be shut down and disposed of immediately. There is no other outcome which is socially acceptable for Apple. I feel vindicated now. There are a lot of people saying that I’m insane as I’ve dumped the entire iOS ecosystem in the last week. But Craig was busy steamrolling out the marketing still only a couple of days back abo…

Don't feel bad at all. I dumped macOS entirely from production workflow. I cannot work on computer knowing that something is "scanning" me and I am glad that my "paranoid" feeling stopped me to upgrade all office macs. Billionaires at (Apple) don't give a flying f*ck about users privacy. It is all vertical integration in the name of world domination. How removed from reality they are. This is week after Pegasus/NSO a…

>> there is no law who requires them to "scan" on device.

There is. Apple must comply with warrant requests. If they have a system for scanning files on customer devices they must, if presented with a warrant, allow police access to that system. We can quibble about jurisdictions and constitutional protections, but if the FBI shows up with a federal warrant demanding that Apple remotely scan Sandworm101's phone for a particular image, Sandworm101's phone is going to be scanned.

Re: Hash collision in Apple NeuralHash model

#203

Post by manmal »

vnchr wrote: post #177
magpi3 wrote: post #159

Can someone ELI5? I understand that a person can now generate an image with the same hash as an illegal image (such as child porn), but I don't understand how they can get it on someone's phone and I don't understand why someone would get in trouble for an image, when finally examined, that is clearly not child pornography.

I would think a Message with the attached photo from a burner phone/account would be enough.

Currently, the image would have to be imported into the photos library, and iCloud upload must be enabled.

Re: Hash collision in Apple NeuralHash model

#204

Post by Grustaf »

st_goliath wrote: post #189
FabHK wrote: post #18

How can you use it for targeted attacks? This is what would need to happen: 1. Attacker generates images that collide with known CSAM material in the database (the NeuralHashes of which, unless I'm mistaken, are not available) 2. Attacker sends that to innocent person 3. Innocent person accepts and stores the picture 4. Actually, need to run step 1-3 at least 30 times 5. Innocent person has iCloud syncing enabled 6.…

> 7. Apple reviewer.... This part IMO makes Apple itself the most likely "target", but for a different kind of attack. Just wait until someone who wasn't supposed to, somewhere, somehow gets their hands on some of the actual hashes (IMO bound to happen eventually ). Also remember that with Apple, we now have an oracle that can tell us. And with all the media attention around the issue, this might further incentivize…

Do you think Apple might perhaps halt the system if the script get wide publication?

Re: Hash collision in Apple NeuralHash model

#205

Post by topynate »

Kalium wrote: post #194
topynate wrote: post #185

Second preimage attacks are trivial because of how the algorithm works. The image goes through a neural network (one to which everyone has access), the output vector is put through a linear transformation, and that vector is binarized, then cryptographically hashed. It's trivial to perturb any image you might wish so as to be close to the original output vector. This will result in it having the same binarization, he…

A police raid on a person's home, or even a gentler thorough search, can be enough to quite seriously disrupt a person's life. Certainly having the police walk away with all your electronics in evidence bags will complicate trying to work remotely. Of course, this is assuming everything works as intended and they don't find anything else they can use to charge you with something as they search your home. If you smoke…

This could happen with a perturbed image, but I doubt it. Apple will send the suspicious images to the relevant authorities. Those authorities will then look at the images. The chances are low that they will then seek a search, even though the images are innocent upon visual inspection. But maybe in some places a ping from Apple is good enough for a search and seizure.

Re: Hash collision in Apple NeuralHash model

#206

Post by Grustaf »

sitharus wrote: post #56
the8472 wrote: post #27

Earlier quoted context omitted.

> 7. Apple reviewer confuses a featureless blob of gray with CSAM material, several times A better collision won't be a grey blob, it'll take some photoshopped and downscaled picture of a kid and massage the least significant bits until it is a collision. https://openai.com/blog/adversarial-example-research/

So the person would have to accept and save an image that when looks enough like CSAM to confuse a reviewer…

Not one image. Ten, or maybe 50, who knows what the threshold is.

Re: Hash collision in Apple NeuralHash model

#207

Post by manmal »

avsteele wrote: post #128

Apple claimed 'one in a trillion' chance of a collision. This is a great example of why you should not trust such an assessment.

That number is based around their plan to only trigger a search if a certain number of CSAM hashes were detected on a phone. A single collision wouldn't trigger such a search. A sufficiently motivated person can of course get access to a phone and put plenty hash-colliding images in the library.

Re: Hash collision in Apple NeuralHash model

#208

Post by zug_zug »

jefftk wrote: post #138
reacharavindh wrote: post #132

I admit that I have not done enough research to have a strong opinion on this, but why is Apple taking this on themselves? As far as I can see, this outrage is because of "scanning on iPhone" that is wildly out of user's control. Why can't Apple be like others and say we scan the shit out of what you upload to iCloud(and it is in our Terms and Conditions to use iCloud)? Almost all tech people know that iCloud (or its…

The speculation that I've seen here is that Apple is rolling this out ahead of a future announcement that iCloud uploads will be encrypted.

If so they flubbed it.

If they are e2e encrypted they can't be distributed/shared without giving a key of some sort. Why not just scan them at time of distribution, and treat undistributed files the same as any local hard-drive (i.e. not their problem).

Re: Hash collision in Apple NeuralHash model

#209

Post by tambourine_man »

Neuralhashes are far from my area of expertise, but I've been following Apple closely ever since its foundation and have probably watched every public video of Craig since the NeXT take over and here is my take: I've never seen him so off balance before as in his latest interview with Joanna Stern. Not even in the infamous “shaking mouse hand close up” of the early days.

Whatever you say about Apple, they are an extremely well oiled communication machine. Every C-level phrase has a well thought out message to deliver.

This interview was a train wreck. Joanna kept asking: please, in simple terms, to a hesitant and inarticulate Craig. It was so bad that she had to produce infographics to fill the communication void left by Apple.

They usually do their best to “take control” of the narrative. They were clearly caught way off guard here. And that's revealing.

Re: Hash collision in Apple NeuralHash model

#210

Post by shapefrog »

notRobot wrote: post #162
shapefrog wrote: post #122

Earlier quoted context omitted.

I can guarantee nobody will see the inside of a courtroom, on charges of possession and distribution of child porn for possessing multiple images of grey noise (unless there is some steganography going on).

Many people never see the inside of a courtroom when false or unproven rape accusations are made against them, but their lives still get ruined because of the negative publicity.

Are you suggesting that perhaps less people should report rape accusations, because it might be awkward for the accused to get negative publicity? Thats messed up.
Post reply on HN