Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

421–430 of 725 posts

Re: Hash collision in Apple NeuralHash model

#421

Post by newsclues »

robertoandred wrote: post #342
newsclues wrote: post #108

First CP. Then leaked or unauthorized nudes will get filtered. Filters for terrorists, and terrorist imagery or symbols. Start scanning for guns and drugs. Drug dealers and criminals get added to the list. How long before before it’s dissidents, political opponents, and minorities in dictatorships? How long before Tim Cooks CP filters are used against LGBT groups abroad?

Do you know how many billions of pictures of guns and drugs have ever been taken?

Sure! Lots!

My Instagram account is mainly weed (legal in Canada) and Instagram censorship hits my content occasionally (because illegal cannabis selling on the platform is rampant).

Re: Hash collision in Apple NeuralHash model

#422

Post by jakear »

Baseless speculation: this is less about ongoing protection against keeping CSAM on their servers and more about a one-time sting operation requested by some sort of acronym’d official. The basic idea being: have a bunch of catfish agents send out these known CSAM images to folks through anonymous channels (sourcing the targets would likely be unfortunately trivial), expect that some of them will save the images and be synced to iCloud, coerce Apple into letting them see who has the material, sting them.

To this end, I would not be at all surprised to see that in some not-too-distant future Apple issues a big ol’ public apology and removes this feature. The operation will of course be long complete by then.

Just writing this out here so I have a “I told you so” link for if/when that time comes :)

Re: Hash collision in Apple NeuralHash model

#423

Post by bumblebritches5 »

magpi3 wrote: post #155
bo1024 wrote: post #152

Earlier quoted context omitted.

Not complete answers but background: apple’s system works by having your device create a hash of each image you have. The hash (a short hexadecimal string) is compared to a list of known CP image hashes, and if it matches, then your image is uploaded to Apple for further investigation. A devastating scenario for such a system is if an attacker knows how to look at a hash and generate some image that matches the hash,…

But how would the attacker get the generated image on a person's phone?

Seriously?

Look into Pegasus and Candiru

Re: Hash collision in Apple NeuralHash model

#424

Post by newsclues »

zug_zug wrote: post #230
newsclues wrote: post #108

First CP. Then leaked or unauthorized nudes will get filtered. Filters for terrorists, and terrorist imagery or symbols. Start scanning for guns and drugs. Drug dealers and criminals get added to the list. How long before before it’s dissidents, political opponents, and minorities in dictatorships? How long before Tim Cooks CP filters are used against LGBT groups abroad?

It's funny how it's always CP or terrorism we need to watch out for.... Never is it the politician who accidentally went to war with wrong country, or the tax return of the congressman, or a cop sleeping with somebody who's under arrest, or EPA employee who took a bribe to declare a chemical safe, or mysteriously malfunctioning cameras in a prison the night of an alleged suicide. Build surveillance to catch those peo…

I am not surprised censorship is used selectively by the powerful against their enemies.

Re: Hash collision in Apple NeuralHash model

#425

Post by topynate »

gowld wrote: post #414
topynate wrote: post #376

Earlier quoted context omitted.

On reflection, yes, there must be warrants involved. I'm raising my estimate of how likely it is that innocent people get raided due to this. The warrant would properly only be to search iCloud, not some guy's house, but I can easily see overly-broad warrants being issued.

> The warrant would properly only be to search iCloud, iCloud is encrypted, so that warrant is useless. They need to unlock and search the device.

Yes, it's encrypted, but part of this anti-CSAM strategy is a threshold encryption scheme that allows Apple to decrypt photos if a certain number of them have suspicious hashes.

Re: Hash collision in Apple NeuralHash model

#426

Post by Invictus0 »

manmal wrote: post #203
vnchr wrote: post #177

Earlier quoted context omitted.

I would think a Message with the attached photo from a burner phone/account would be enough.

Currently, the image would have to be imported into the photos library, and iCloud upload must be enabled.

Whatsapp has a feature where all images are automatically saved to device as they are received. If automatic iCloud upload is on, then all the conditions are there for a person to innocently click on a spammy Whatsapp message, see a bunch of nonsense grayscale images, and continue on with their day--not realizing they are now being monitored for CSAM.

Re: Hash collision in Apple NeuralHash model

#427

Post by snowwrestler »

sandworm101 wrote: post #402
snowwrestler wrote: post #389

Earlier quoted context omitted.

Can you explain why you think CALEA would apply to what Apple announced? And why such application would have had to wait until Apple announced this? In other words, if the law can force Apple to do things in general, why does the law need to wait for Apple to announce certain capabilities first?

>> why you think CALEA would apply to what Apple announced? Read what I stated. I said no such thing. I said that CALEA shows that companies can sometimes be forced to be do things they do not want to do, to take actions at the behest of law enforcement that they would not normally do. CALEA would clearly not be applied to apple in this case. It would be some other law/warrant/NSL that would force apple to do somethi…

I understand why you are worried that a more powerful law could be passed by Congress in the future.

I thought this particular conversation was about what Apple can be forced to do by a warrant issued under current law.

Re: Hash collision in Apple NeuralHash model

#428

Post by stetrain »

short_sells_poo wrote: post #404
stetrain wrote: post #341

Earlier quoted context omitted.

I don't think "All user data on device and in the cloud is not scannable for CSAM or retrievable with a warrant" is a tenable position in the current US political landscape, and even less so in other countries. And my point is that if the government wanted a dragnet they could just legislate or secretly order one. Just like they have done in various forms over the last 20 years. And Apple might not even be allowed to…

> "All user data on device and in the cloud is not scannable for CSAM or retrievable with a warrant" Who said anything about warrants? As far as I know the proposed system is proactive and requires no warrants at all. More to the point, anyone remotely sophisticated can just encrypt CSAM into a binary blob and plaster it all over the cloud providers servers. Ie, this system will possibly catch some small time pervert…

I haven’t ever said this is a good thing and that we should like it.

I’m saying if the concern is that a government orders Apple to change it and do something different, then that’s a government problem and maybe we should try fixing that.

Re: Hash collision in Apple NeuralHash model

#429

Post by brandon272 »

HatchedLake721 wrote: post #383
tambourine_man wrote: post #209

Neuralhashes are far from my area of expertise, but I've been following Apple closely ever since its foundation and have probably watched every public video of Craig since the NeXT take over and here is my take: I've never seen him so off balance before as in his latest interview with Joanna Stern. Not even in the infamous “shaking mouse hand close up” of the early days. Whatever you say about Apple, they are an extr…

Can you link the interview please?

I assume it is this one: https://www.youtube.com/watch?v=OQUO1DSwYN0

This was painful to watch.

Re: Hash collision in Apple NeuralHash model

#430

Post by [deleted] »

robertoandred wrote: post #379
Rd6n6 wrote: post #143

Earlier quoted context omitted.

That image planting virus concept is the scariest thing. Could you trigger this system and get the police alerted with spam mms with images or spam email with images, or targeted ads that get the gray blob images into your tmp files? Or does it have to be an actual script downloading images, say, one per week in secret until it triggers?

No, you couldn't. This is only checking images you upload to your iCloud photo library. Why would you save tons of gray blobs to your photo library? Why would gray blobs look like child porn? Why would Apple reviewers think gray blobs are child porn? Why would the NCMEC think gray blobs are child porn? Why would law enforcement spend time arresting someone for gray blobs?

[deleted]
Post reply on HN