Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

341–350 of 725 posts

Re: Hash collision in Apple NeuralHash model

#341

Post by stetrain »

short_sells_poo wrote: post #334
stetrain wrote: post #258

Earlier quoted context omitted.

This exactly. I am mostly convinced based on the technical details that have (slowly) come out from Apple that they have made this system sufficiently inconvenient to use as a direct surveillance system by a malicious government. Yes a government could secretly order Apple to make changes to the system, but they could also order them to just give them all of your iCloud photos and backups, or send data directly from…

Yes of course it is more private than outright scanning all photos but how does that relate to the ground state of private photos simply not being scanned at all? And please do not bring in whataboutist arguments like "but other cloud providers are already doing it". I'm honestly taken aback by how many people on HN are completely OK with what Apple is pulling here. In my mind, it's irrelevant that the current system…

I don't think "All user data on device and in the cloud is not scannable for CSAM or retrievable with a warrant" is a tenable position in the current US political landscape, and even less so in other countries.

And my point is that if the government wanted a dragnet they could just legislate or secretly order one. Just like they have done in various forms over the last 20 years. And Apple might not even be allowed to tell us it is happening.

Re: Hash collision in Apple NeuralHash model

#342

Post by robertoandred »

newsclues wrote: post #108

First CP. Then leaked or unauthorized nudes will get filtered. Filters for terrorists, and terrorist imagery or symbols. Start scanning for guns and drugs. Drug dealers and criminals get added to the list. How long before before it’s dissidents, political opponents, and minorities in dictatorships? How long before Tim Cooks CP filters are used against LGBT groups abroad?

Do you know how many billions of pictures of guns and drugs have ever been taken?

Re: Hash collision in Apple NeuralHash model

#343

Post by bo1024 »

brokensegue wrote: post #327
bo1024 wrote: post #147

Earlier quoted context omitted.

> The fact that you can randomly manipulate random noise until it matches the hash of an arbitrary image is not surprising. Strongly disagree. (1) The primary feature of any decent hash function is that this should not happen. (2) Any preimage attack opens the way for further manipulations like you describe.

cryptographic hashes are different from image fingerprints

That's true, one way to put it is that traditionally non-cryptographic hashes are supposed to prevent accidental collisions, while cryptographic ones should prevent even collisions on purpose.

But hashing is used in many places that could be vulnerable to an attack, so I think the distinction is blurry. People used MD5 for lots of things but are moving away for this reason, even though they're not in cryptographic settings.

Re: Hash collision in Apple NeuralHash model

#344

Post by short_sells_poo »

Grustaf wrote: post #266
sandworm101 wrote: post #246

Earlier quoted context omitted.

>> useful for areas where Apple really doesn't want to even look at the actual material Correct. It has plausible deniability built in. Apple is unable to verify that the images the government are looking for are actually CSAM. They could be political. They could be protest images. They could be Winnie the Pooh. Apple can plead ignorance as it blindly scans for whatever the requesting government asks it to scan for.…

Except that Apple will review the photos once you've matched 30 of them, so it's still not possible for the government to misuse it.

So some underpaid contractor reviewing "visual derivatives" that may or may not be CSAM completely prevents governments from misusing this in your mind?

Re: Hash collision in Apple NeuralHash model

#345

Post by argvargc »

hughrr wrote: post #7

That’s end game. Now you can use it for targeted attacks against innocent people. This needs to be shut down and disposed of immediately. There is no other outcome which is socially acceptable for Apple. I feel vindicated now. There are a lot of people saying that I’m insane as I’ve dumped the entire iOS ecosystem in the last week. But Craig was busy steamrolling out the marketing still only a couple of days back abo…

It didn't even last a week.

Introducing the people asking us to trust them with the responsibility of mass, automated crime accusations.

Re: Hash collision in Apple NeuralHash model

#346

Post by proactivesvcs »

Many of us have been at the sharp end of a large company's supposed "human review" process when it comes to account lockouts, invalid abuse reports and invalid accusations of breach of terms of service. It simply does not work for too many and leaves us with little or no redress; it is a completely untenable fallback.

When you're promised that this will never happen to you because there are humans in the process remember that they're one or more of underpaid, poorly-trained, poorly-treated, under-resourced or just flat-out terrible at their job.

This may well happen to you and you'll have no way back.

Re: Hash collision in Apple NeuralHash model

#347

Post by falcolas »

topynate wrote: post #205
Kalium wrote: post #194

Earlier quoted context omitted.

A police raid on a person's home, or even a gentler thorough search, can be enough to quite seriously disrupt a person's life. Certainly having the police walk away with all your electronics in evidence bags will complicate trying to work remotely. Of course, this is assuming everything works as intended and they don't find anything else they can use to charge you with something as they search your home. If you smoke…

This could happen with a perturbed image, but I doubt it. Apple will send the suspicious images to the relevant authorities. Those authorities will then look at the images. The chances are low that they will then seek a search, even though the images are innocent upon visual inspection. But maybe in some places a ping from Apple is good enough for a search and seizure.

FWIW, they won't send the images. Even in the pursuit of knocking back CSAM, there are strict restrictions on the transmission and viewing of CSAM - in some cases even the defendant's lawyers don't usually see the images themselves in preparation for a trial, just a description of the contents. Apple employees or contractors will likely not look at the images themselves, only visual hashes.

They will instead contact the police and say "Person X has Y images that are on list Z," and let the police get a warrant based off that information and execute it to check for actual CSAM.

Re: Hash collision in Apple NeuralHash model

#348

Post by bo1024 »

Edd314159 wrote: post #181
bo1024 wrote: post #152

Earlier quoted context omitted.

Not complete answers but background: apple’s system works by having your device create a hash of each image you have. The hash (a short hexadecimal string) is compared to a list of known CP image hashes, and if it matches, then your image is uploaded to Apple for further investigation. A devastating scenario for such a system is if an attacker knows how to look at a hash and generate some image that matches the hash,…

> A devastating scenario for such a system is if an attacker knows how to look at a hash and generate some image that matches the hash, allowing them to trigger false positives any time. This is my understanding too. But is this not also true for other (cloud-based) CSAM scanning systems? Why is Apple's special in this regard?

I don't know. I don't know what other systems use, I just know what I've read recently about Apple's.

Re: Hash collision in Apple NeuralHash model

#349

Post by treis »

ethbr0 wrote: post #248
sandworm101 wrote: post #201

Earlier quoted context omitted.

>> there is no law who requires them to "scan" on device. There is. Apple must comply with warrant requests. If they have a system for scanning files on customer devices they must, if presented with a warrant, allow police access to that system. We can quibble about jurisdictions and constitutional protections, but if the FBI shows up with a federal warrant demanding that Apple remotely scan Sandworm101's phone for a…

That's an incomplete statement. Currently, they must comply with warranty requests by scanning if they have the ability to scan . If they have no such ability (say, because they designed their phones from a privacy-first perspective), the law makes no requirement that they create such a capability. And that's what pisses people off about this.

Warrants are only part of the issue. The bigger issue is civil and criminal liability. If someone uploads child porn to iCloud and then shares it with someone else Apple is possessing and distributing child porn.

Today they aren't liable because there is a law that shields them. But that law comes with strings attached around assisting law enforcement. By doing an end run around those strings Apple is not holding up it's end of the bargain and runs the risk of lawmakers removing the liability shield.

If that happens then it's a whole new ball game.

Re: Hash collision in Apple NeuralHash model

#350

Post by lumost »

tantalor wrote: post #286
ethbr0 wrote: post #248

Earlier quoted context omitted.

That's an incomplete statement. Currently, they must comply with warranty requests by scanning if they have the ability to scan . If they have no such ability (say, because they designed their phones from a privacy-first perspective), the law makes no requirement that they create such a capability. And that's what pisses people off about this.

Can a warrant compel them to develop the capability?

The law gets debatable here. The warrant can ultimately be served only to the owner/responsible party for the system. If apple decides that they own all iPhones, a claim they've loosely made for a long time re Flash etc. Then they could be served a warrant on the device.

If they just sold the device to someone, then the police would have to issue a warrant to that individual. My understanding is that as an individual your options are either to comply with the warrant or commit a separate crime destroying the evidence or refusing the warrant.

Post reply on HN