Live data from Hacker News

Apple’s new abuse prevention system: an antritust/competition point of view

blog.quintarelli.it

171–180 of 318 posts

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#171
post #62
post #45

Earlier quoted context omitted.

This is a great point. You don't even need to unlock an iPhone to take a picture. So in theory anyone with access to your phone for a few seconds could incriminate you with little effort.

This is already possible today with things like iCloud Photos, Google Photos, OneDrive etc.

Today: "WTF is this?" delete

iOS 15: "WTF is this?" SWAT team crashes through window

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#172
post #144

Earlier quoted context omitted.

No. You can design a system where the FPR is essentially zero. Even if shitty md5 is used.

> No. You can design a system where the FPR is essentially zero. Even if shitty md5 is used. How can you do that, considering md5 can have collisions?

A confusing thing that I think people haven't addressed clearly for the most part:

For a hash (whether cryptographic or perceptual), there is a chance of random collisions and also a difficulty factor for adversarially-created intentional collisions. The random collision probability has to be estimated based on some model of the input and output space (with cryptographic hash functions, you would usually model them as pseudorandom functions and assume that the collision probability is the same one created by the birthday paradox calculation).

Intentional collisions depend on insight about the structure of the hash function, and there are also different kinds of difficulty levels depending on the nature of the attack (preimage resistance, second-preimage resistance, and collision resistance). Gaining more insight about the structure of the hash function can act to reduce the work factor required for mounting these attacks. That should be true for perceptual hashes just as much as cryptographic hashes, but presumably all of the intentional attacks should start off easier because the perceptual hashes' threat models are weaker and there's much less mathematical research on how to achieve them.

And in AI systems involving classifiers, it was generally easy for people to create adversarial examples given access to the model. Perceptual hashes for estimating similarity to specific known images aren't the exact same thing because it's less like "how much like a cat is this image?" and more like "how much like NCMEC corpus image 77 is this image?", but maybe some of the same techniques would still work. In the cryptographic hash analogy, I guess that would be like trying to break preimage resistance.

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#173
post #145

Earlier quoted context omitted.

Yes but my understanding is that the hashes are perceptual, as opposed to cryptographic. If the system was matching against known cryptographic hashes the collision / false positive rate would be small, but the fuzzy matching involved with perceptual hashing necessarily has a greater false positive rate. And that doesn’t even begin to address the detection of sent and received “explicit images” which are detected on…

I suspect that's why they have some threshold that moves the false positive rate to one in one trillion. The iMessage bit is different - it's only on device, only on child accounts, and only alerts parents. It's more akin to a parental control feature than anything else.

This is a great point, thanks for adding that detail.

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#175
post #146

Earlier quoted context omitted.

If you actually think there is going to be a fully automated system dispatching police SWAT teams throughout the US without a manual (or judicial) review then.... I really don't know what to tell you.

Yeah, it normally requires an anonymous phone call from a VoIP number.

Is this an allusion to an actual occurrence?

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#176

Earlier quoted context omitted.

Yeah, it normally requires an anonymous phone call from a VoIP number.

Is this an allusion to an actual occurrence?

https://en.wikipedia.org/wiki/Swatting

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#177
post #88

Earlier quoted context omitted.

> It'll start with protecting children. We all want to protect children, don't we? Why do you want children abused? Are you a child abuser, what do you have to hide? To be clear, this is continued enforcement for years-old regulation. The feature is only enabled in the US where it is required. The implementation is changing from cloud-based matching (which requires photos to be readable by their cloud infrastructure)…

Could you point to the regulation that requires this?

See the “Federal CSAM Law” section here:

https://cyberlaw.stanford.edu/blog/2020/01/earn-it-act-how-b...

> Section 2258A of the law imposes duties on online service providers, such as Facebook and Tumblr and Dropbox and Gmail. The law mandates that providers must report CSAM when they discover it on their services, and then preserve what they’ve reported (because it’s evidence of a crime).

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#178
post #28

There is no such thing as a trustworthy third party and even trusting yourself is questionable at the best of times. We are constantly balancing a bunch of different considerations with regards to the way that we compute, purchase devices, and utilize services. Security and privacy are of course important, and Apple to date has had a fairly good (if shallow) track record in this regard, at least in the United States.…

> "As mentioned in the article, this does absolutely nothing towards protecting children other than directing all but the biggest idiots towards platforms that can't be linked to them, which I'd imagine, they already are." I suspect you're more wrong than you think about this. People share large volumes of CSAM through lots of different services - I knew someone who worked on the problem at Linked In (!). HN likes to…

It does not imply a false zero positive rate at all, because the hashes used are by necessity fuzzy, and even a completely benign picture can trigger a match.

Now they're saying that if the match is a false positive, it'll be screened by the human reviewer. But that means there's some stranger there potentially looking at my private photos (and, by the way, I wonder which country they'll be located in?). That's already completely and utterly unacceptable - you don't have to wait for any "future complications".

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#179
post #108

Earlier quoted context omitted.

It's not a math problem, it's a human problem. suppose you have a partner who is a 'petite' woman of 34. She enjoys posting nudies on a website, but without her face in that picture. Someone who collects child porn downloads it, because he enjoys that picture. A year later he gets caught by the police and all his pictures get marked as 'verified child porn'. Suddenly you get marked as owning child porn.

That isn’t CSAM. https://www.nytimes.com/interactive/2019/09/28/us/child-sex-... Apple’s thing has some sort of threshold anyway so one image would not trigger it. I don’t buy your example - the CSAM images are not what you’re describing.

Who determines whether something is CSAM? How do you or I know that every single one of those images actually is CSAM? How do we know if the FBI or CIA or CCP adds hashes of innocent pics in order to pin a crime against someone?

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#180
post #35

Apple will release this in their desktop pcs. The major consequence of this is that, a year down the line, microsoft will also be compelled/forced to join this "coalition of the good". After all, they already scan all your files for viruses, it would be a shame if they didn't scan for anything that is deemed incriminating and also call the cops on you. Of course, the children are being used as a trojan horse again. W…

>We don't even mention the giant logical leap from finding someone possessing CP to automatically considering them a child molester, yet someone posessing an action movie is not considered a murderer. This is a wild and in my opinion a wrongheaded analogy. Possessing CP is a crime by itself. It doesn't matter if the person possessing is actually a molester or not. It is just like the possession of drugs being illegal…

The worst thing about it is that CP possession is a strict liability crime in most jurisdictions. Meaning that prosecution doesn't have to prove intent - if they can find it on your machine, you're guilty regardless of how it got there.
Post reply on HN