Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple’s new abuse prevention system: an antritust/competition point of view

blog.quintarelli.it

Post reply on HN
Print view View original

141–150 of 318 posts

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#141

Post by flutas »

gjsman-1000 wrote: post #90
flutas wrote: post #73

Earlier quoted context omitted.

> Someone who posesses an action movie likes action movies, while someone who posesses child porn likes child porn. Unless it's planted. [0] Or sent to you. [1] Or (farther out there) happens to be embedded on a site you visited and ends up in your browser cache. [0]: https://www.nytimes.com/2016/12/09/world/europe/vladimir-put... [1]: https://www.nytimes.com/2019/06/17/nyregion/alex-jones-sandy...

It only scans images in your iCloud Photo Library. Not paying for iCloud? No scanning. Not in your photo library? No scanning. And even then, only for known content, not new content.

Conveniently, iOS 15 also syncs images from your messages and many other places into your photos. Whether this will put that file in your iCloud Photo library, I do not know.

https://www.macrumors.com/how-to/see-photos-shared-with-you-...

On top of that, at this stage you are right. How long before they move it to every file in your device's storage "because of the children!"

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#142

Post by 2pEXgD0fZ5cF »

DSingularity wrote: post #121
2pEXgD0fZ5cF wrote: post #98

Earlier quoted context omitted.

> the false positive rate is essentially zero I highly doubt that

Based on what? Unless you point out a flaw in the math, it’s zero.

> Based on what?

Based on the fact that ultimately we can't check the system. And based on the fact that at some point in the chain humans are involved. [1]

What we are left with is "trust in Apple" not "trust in math".

[1]: https://news.ycombinator.com/item?id=27878333

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#143

Post by nomel »

gjsman-1000 wrote: post #87
734129837261 wrote: post #51

It'll start with protecting children. We all want to protect children, don't we? Why do you want children abused? Are you a child abuser, what do you have to hide? Next it's elderly people. We don't want our forgetful elders to get lost, do we? What if grandma wanders off but is in someone's picture, surely you want the police to know right that second where she is? Next up, terrorists! Four adult brown men in an unm…

Remember that it only scans against a known database of already found content, and does not try to find new content.

Is this true? The message triggers try to identify nudity within the accounts of minors. Is the notification only going to the parent? Is it stored for possible later use? Are those photos ever reviewed?

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#144

Post by Zababa »

DSingularity wrote: post #119
paul_f wrote: post #93

Earlier quoted context omitted.

This seems naive, there are always false positives

No. You can design a system where the FPR is essentially zero. Even if shitty md5 is used.

> No. You can design a system where the FPR is essentially zero. Even if shitty md5 is used.

How can you do that, considering md5 can have collisions?

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#145

Post by bnj »

gonehome wrote: post #91
lamontcg wrote: post #60

> But when a backdoor is installed, the backdoor exists and history teaches that it’s only a matter of time before it’s also used by the bad guys and authoritarian regimes. Problem is that this scanning is necessarily fuzzy and there is going to be a false positive rate to it. And the way that you'll find out that you've tripped a false positive is that the SWAT team will knock your door down and kill your dog (at a…

This is wrong - the iCloud check is against known CSAM hashes, the false positive rate is essentially zero.

Yes but my understanding is that the hashes are perceptual, as opposed to cryptographic.

If the system was matching against known cryptographic hashes the collision / false positive rate would be small, but the fuzzy matching involved with perceptual hashing necessarily has a greater false positive rate.

And that doesn’t even begin to address the detection of sent and received “explicit images” which are detected on device and don’t have a set of known hashes.

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#146

Post by mthoms »

mikewarot wrote: post #135
mthoms wrote: post #113

Earlier quoted context omitted.

> Problem is that this scanning is necessarily fuzzy and there is going to be a false positive rate to it. And the way that you'll find out that you've tripped a false positive is that the SWAT team will knock your door down and kill your dog (at a minimum). Not true. Hash matches are to be human reviewed. So no, people won't get "swatted" accidentally as you allege. The other concerns people have been voicing are ce…

>Hash matches are to be human reviewed. Until it proves too expensive, then a different AI system will do it instead. I have zero faith that it'll be a fully competent, well trained, well rested, well paid person will actually be doing these reviews in the long run.

If you actually think there is going to be a fully automated system dispatching police SWAT teams throughout the US without a manual (or judicial) review then.... I really don't know what to tell you.

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#147

Post by patmcc »

hhh wrote: post #92
gonehome wrote: post #91

Earlier quoted context omitted.

This is wrong - the iCloud check is against known CSAM hashes, the false positive rate is essentially zero.

One in one trillion chance per year, per the paper on the Apple site.

I see the 'one in one trillion' but it's a bit vague; I read it as '1 in trillion chance' per image.

So when we have a billion iPhones in the wild taking 10 images a day...1 in a trillion chances happen every few months. Now, if that triggers some further review, maybe that's an acceptable false positive. If it triggers a SWAT team, I don't think it is.

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#148

Post by nomel »

mthoms wrote: post #131
artificial wrote: post #63

Earlier quoted context omitted.

The government controls the hash list.

No. A registered charity called The National Center for Missing and Exploited Children controls the hash list. Yes, they are partly government funded, but I highly doubt they'd let their mission be compromised by allowing the government to inject non-CP hashes. Doing so would compromise all the work they've performed over the last four decades. These people are (rightfully) very passionate about their work and can't…

> by allowing the government to inject non-CP hashes

I don't think "allowing" is the concern here, because I highly doubt they get to generate the hashes themselves.

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#149

Post by gonehome »

bnj wrote: post #145
gonehome wrote: post #91

Earlier quoted context omitted.

This is wrong - the iCloud check is against known CSAM hashes, the false positive rate is essentially zero.

Yes but my understanding is that the hashes are perceptual, as opposed to cryptographic. If the system was matching against known cryptographic hashes the collision / false positive rate would be small, but the fuzzy matching involved with perceptual hashing necessarily has a greater false positive rate. And that doesn’t even begin to address the detection of sent and received “explicit images” which are detected on…

I suspect that's why they have some threshold that moves the false positive rate to one in one trillion.

The iMessage bit is different - it's only on device, only on child accounts, and only alerts parents. It's more akin to a parental control feature than anything else.

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#150

Post by mikewarot »

mthoms wrote: post #146
mikewarot wrote: post #135

Earlier quoted context omitted.

>Hash matches are to be human reviewed. Until it proves too expensive, then a different AI system will do it instead. I have zero faith that it'll be a fully competent, well trained, well rested, well paid person will actually be doing these reviews in the long run.

If you actually think there is going to be a fully automated system dispatching police SWAT teams throughout the US without a manual (or judicial) review then.... I really don't know what to tell you.

It'll all be shadow-ban type stuff, your account will be turned off, without appeal, things of that nature.
Post reply on HN