Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple enabling client-side CSAM scanning on iPhone tomorrow

twitter.com

Post reply on HN
Print view View original

731–740 of 757 posts

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#731

Post by ddtaylor »

hda2 wrote: post #585
bigiain wrote: post #27

Earlier quoted context omitted.

> and they want to check if any of the hashes are the same as hashes of child porn? ... without any technical guarantee or auditability that any of the hashes they're alerting on are actually of child porn. How much would you bet against law enforcement to abuse their ability to use this, and add hashes to find out who's got anti government memes or police committing murder images on their phones? And that's just in…

I remember the story where some large gaming company permanently banned someone because they had a file with a hash that matched a "hacking tool". Turns out the hash was for an empty file. This will end badly for humanity.

Path of Exile has in the past looked at DNS cache information on the system and will ban players who have contacted known bad sites.

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#732

Post by Aeolun »

Filligree wrote: post #503
Aeolun wrote: post #488

Earlier quoted context omitted.

I’m not sure this is true? The $1000 version has absolutely ridiculous performance for it’s price class. To the point it’s nearly as good as my desktop system.

My desktop system is a 24-core Zen 2. The M1 shouldn't be faster, and I'm certain the difference is almost purely a matter of software, but in reality the M1 certainly feels a lot faster. Yes, the desktop has higher throughput. Of course it does. But that doesn't mean I don't feel a fraction of a second's lag whenever I do basically anything, and on the M1 that just... doesn't exist.

On the other hand, my i7 16 inch MacBook Pro consists of purely lag. Half of the time it’ll be the ‘kernel process’ so I don’t even have a clue what’s causing it.

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#733

Post by headmelted »

ok123456 wrote: post #610
andrewmd5 wrote: post #481

Earlier quoted context omitted.

If you can recreate a file so it’s hash matches known CP then that file is CP my dude. The probability of just two hashes accidentally colliding is approximately: 4.3*10-60 Even if you do a content aware hash where you break the file into chunks and hash each chunk, you still wouldn’t be able to magically recreate the hash of a CP file without also producing part of the CP.

It's NOT a cryptographic hash. It's the weights from the middle of a neural network that they're calling a "hash" because it encodes and generates an image it has classified as bad. Experts have trouble rationalizing about what weights mean in a neural network. This is going to end badly.

Exactly this.

If this was a hash then it would be as the parent describes, this is at best a very fuzzy match on an image to take into account blurring/flipping/colour shifting.

It's vastly more likely that innocent people will be implicated for fuzzy matches on innocuous photos of their own children in shorts/swimming clothes than it is to catch abusers.

The other thing is, when you have nothing to hide you won't take efforts to hide it - meaning you'll upload all of your (completely normal) photos to iCloud without thinking about it again.

The monsters making these images know what they're doing is wrong, so they'll likely take efforts to scramble or further encrypt the data before uploading.

tldr; it's far likelier that this dragnet will only even apply to innocent people, than it is to catch predators.

All this said, I'm still in support of Apple taking steps in this direction, but it needs far more protections put in place to prevent false positives than this solution allows. A single false accusation by this system, even if retracted later and rectified, would destroy an entire family's lives (and could well cause suicides).

Look what happened in the Post Office case in the UK as an example of how these things can go wrong - scores of people went to prison for years for crimes they didn't commit because of a simple software bug.

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#734

Post by bantunes »

quenix wrote: post #725
marderfarker2 wrote: post #386

Earlier quoted context omitted.

Current generation of desktop/laptop processors are plenty powerful. I’m unconvinced they cannot fulfil today’s computing needs.

As an M1 owner, M1 just does it better for mobile computing purposes. There is no denying this.

You know, there was a time when people stopped buying cheap shirts made in English factories that employed kids. They were good shirts, too.

Is it worth it, though? Is being "better for mobile computing purposes" worth all of this, and whatever else will come next - which we know is just a matter of time, because no one cares enough to stop buying from them?

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#735

Post by dragonwriter »

headmelted wrote: post #733
ok123456 wrote: post #610

Earlier quoted context omitted.

It's NOT a cryptographic hash. It's the weights from the middle of a neural network that they're calling a "hash" because it encodes and generates an image it has classified as bad. Experts have trouble rationalizing about what weights mean in a neural network. This is going to end badly.

Exactly this. If this was a hash then it would be as the parent describes, this is at best a very fuzzy match on an image to take into account blurring/flipping/colour shifting. It's vastly more likely that innocent people will be implicated for fuzzy matches on innocuous photos of their own children in shorts/swimming clothes than it is to catch abusers. The other thing is, when you have nothing to hide you won't ta…

> The monsters making these images know what they're doing is wrong, so they'll likely take efforts to scramble or further encrypt the data before uploading.

The ones that make national news from big busts do, because the ones that don't get caught much sooner and only make local news, because Google and other parties are have automatic CSAM identification online already (server side, not client side, AFAIK), and are sending hits to Homeland Security.

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#736

Post by julik »

disiplus wrote: post #217
jsjohnst wrote: post #63

Earlier quoted context omitted.

I have direct knowledge of examples of where individuals were arrested and convicted of sharing CP online and they were identified because a previous employer I worked for used PhotoDNA analysis on all user uploaded images. So yeah, this type of thing can catch bad people. I’m still not convinced Apple doing this is a good thing, especially on private media content without a warrant, even though the technology can he…

now im afraid, i have two young children < 5 years old. i have occasionally took pictures of them naked with some bumps on the skin or mosquito bite and sent them to my wife over whatsapp to look at and decide do we need to send them to doctor, do i have to fear now that i will be marked as distributing CP.

A historian in Russia has been jailed because of comparable charges (while in fact he got pinned hard due to him investigating Stalin-time mass executions). https://abcnews.go.com/International/russia-convicts-histori...

After all the courts he ended up with 13 years in prison, where he is likely going to die.

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#737

Post by roenxi »

junon wrote: post #566
freshdonut wrote: post #534

Earlier quoted context omitted.

> is like saying you know black people and that somehow it affords you some privilege others do not possess. Of course it does. Interacting with black people (or any race) affords you insight into their life experiences, struggles, worldview etc... Of course sociological discourse is highly subjective but this attitude on HN that anecdotal data has no value whatsoever is silly. Do you seriously expect every fact of e…

Was alluding to the common argument of "I can say the N word, I know black people" spiel. You've missed my point, I suspect on purpose. Just knowing someone of a particular demographic doesn't mean you're entitled to generalize about them. That's not how discourse works.

Based on a fairly cursory examination, the "N-word" ban is largely enforced by white people and using it is likely to get them more riled up than black folk.

So simultaneously I can imagine how black people wouldn't care and why that doesn't matter. The word isn't banned because blacks have delicate eardrums, it is banned because white people are showing respect.

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#739

Post by u4918 »

In Apple's public comments they repeatedly say things like "Apple can't access metadata or visual derivatives for matched CSAM images until a threshold of matches is exceeded for an iCloud Photos account."

If they are only concerned with iCloud accounts then... why not scan in the cloud? Can anyone explain to me why client-side scanning is actually needed here? As far as I'm aware, Apple only E2E encrypts iMessages, not iCloud photos or backups.

US politicians (to say nothing of countries with less individual freedoms) already openly pressure tech companies to censor specific content. And tech companies will do so even to the point of blocking direct, private, 1-1 messages sharing specific news. In that light, Apple's crossing a line to client-side scanning seems deeply concerning.

I don't see how keeping this as narrowly-targeted as it's being advertised would ever be possible or even intended.

Re: Apple enabling client-side CSAM scanning on iPhone tomorrow

#740

Post by bagacrap »

bigiain wrote: post #27
ajsnigrutin wrote: post #3

So if I understand correctly, they want to scan all your photos, stored on your private phone, that you paid for, and they want to check if any of the hashes are the same as hashes of child porn? So... all your hashes will be uploaded to the cloud? How do you prevent them from scanning other stuff (memes, leaked documents, trump-fights-cnn-gif,... to profile the users)? Or will a huge hash database of child porn hash…

> and they want to check if any of the hashes are the same as hashes of child porn? ... without any technical guarantee or auditability that any of the hashes they're alerting on are actually of child porn. How much would you bet against law enforcement to abuse their ability to use this, and add hashes to find out who's got anti government memes or police committing murder images on their phones? And that's just in…

they don't check the unhashed bytes against the child porn bytes after a hash match?
Post reply on HN