Apple’s new abuse prevention system: an antritust/competition point of view • Hacker News Board

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#271

Post by skinkestek » Sat, Aug 07, 2021, 4:44 PM UTC

gonehome wrote: post #149

bnj wrote: post #145

Earlier quoted context omitted.

Yes but my understanding is that the hashes are perceptual, as opposed to cryptographic. If the system was matching against known cryptographic hashes the collision / false positive rate would be small, but the fuzzy matching involved with perceptual hashing necessarily has a greater false positive rate. And that doesn’t even begin to address the detection of sent and received “explicit images” which are detected on…

I suspect that's why they have some threshold that moves the false positive rate to one in one trillion. The iMessage bit is different - it's only on device, only on child accounts, and only alerts parents. It's more akin to a parental control feature than anything else.

> I suspect that's why they have some threshold that moves the false positive rate to one in one trillion.

So now instead of sending just one nice innocent very high resolution images of "Tokyo City" or something with something horrific hidden somewhere you have to send a few such images.

That is reassuring. I can never believe anyone except me will think about that.

(If the system is too dumb to detect this it is worthless, and if it is smart enough this opens the floodgates for anyone wanting to make trouble for just about anyone.)

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#272

Post by skinkestek » Sat, Aug 07, 2021, 4:48 PM UTC

nomel wrote: post #140

Conlectus wrote: post #132

Earlier quoted context omitted.

Perceptual hashes are not exact hashes, otherwise they would be useless for this task; you would just mirror the image or change 1 pixel. They are instead fuzzy classifiers, and thus have non-zero error rates.

But this trigger requires many of these false flags to exceed the threshold (most likely why it exists). I imagine the "one in a trillion" numbers they claim for false flagging rate are probably cemented in reality, and make it trivial for human review.

I've explained the problem here: https://news.ycombinator.com/item?id=28099927

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#273

Post by skinkestek » Sat, Aug 07, 2021, 4:50 PM UTC

DSingularity wrote: post #190

Zababa wrote: post #185

Earlier quoted context omitted.

Thanks for the detailed explanation. I understand why that works for perceptual hashes if you make them really precise, however I doubt it would work with md5, which is why I asked.

The discussion I thought we were having was about false positives and not adversarially induced false positives. For the former the random collisions have a probability of 1/(2^64). To mitigate adversarial false positives one idea is to use the combination of a cryptographically strong hash along with a randomly selected perturbation of the file. Prior to hashing, perturb the file and submit both the hash and the sel…

> The discussion I thought we were having was about false positives and not adversarially induced false positives.

I think the rest of us have been discussing how this can and will be abused, by definition by adversaries.

Many of us have also observed for years how systems are abused so we sadly have a gut feeling for this.

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#274

Post by ctvo » Sat, Aug 07, 2021, 4:58 PM UTC

DSingularity wrote: post #192

ctvo wrote: post #139

Earlier quoted context omitted.

I can't find a widely accepted statistic but these numbers should illustrate the point: - A 2016 study by the Center for Court Innovation found that between 8,900 and 10,500 children, ages 13 to 17, are commercially exploited each year in the United States. (Center for Court Innovation, 2016) https://www.courtinnovation.org/sites/default/files/document... - The annual number of persons prosecuted for commercial sexua…

Thanks for the statistics. To me, these numbers are significant. Maybe Whole Foods or maybe some popular restaurants are better candidates for working on improving nutrition in public schools? Why don’t we let apple contribute where it thinks it can. Maybe with apple that number goes down from 10,000 to 2,000. Wouldn’t that be a celebrated outcome?

> To me, these numbers are significant.

What would be insignificant? 1 child? 100? There are 73,000,000 children (under 18) in the US alone. 10,000 is .0001% of that population.

> Why don’t we let apple contribute where it thinks it can.

Apple is the most profitable company in the world. It's a company that prides itself on its imagination and innovation, I wouldn't discount their ability to come up with something.

> Maybe with apple that number goes down from 10,000 to 2,000. Wouldn’t that be a celebrated outcome?

No, it's not. We make trade-offs all the time. The possible harm to Apple's user base is not worth the possibility that this reduces child abuse. There's a possibility these people move on to another platform and this does nothing.

To get that number down Apple creates an entry point for violating the privacy of half a billion users worldwide. Many of them are in China, where pressure from the government has already moved Apple in directions that are harmful to its customers[1].

1 - https://www.nytimes.com/2021/05/17/technology/apple-china-ce...

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#275

Post by anonuser123456 » Sat, Aug 07, 2021, 4:59 PM UTC

int_19h wrote: post #218

anonuser123456 wrote: post #193

Earlier quoted context omitted.

In criminal law you must be aware of an item for it to be under your possession. If someone plants illegal content on your computer, and you are unaware, you aren't legally in possession.

"Typically in criminal law, the defendant's awareness of what he is doing would not negate a strict liability mens rea (for example, being in possession of drugs will typically result in criminal liability, regardless of whether the defendant knows that he is in possession of the drugs)." https://www.law.cornell.edu/wex/strict_liability

Note the word 'typically'.

The purpose of strict liability in possession is to prevent the defense that someone does not know the legal status of an item in their possession. It does NOT prevent the defense that someone does not _know_ something to be in their possession.

For example, it is not a defense to have drugs and claim "but I didn't know they were illegal". It is a defense to claim "I did not know they were there."

In drug cases with actual possession, it is difficult to support a defense of "I didn't know they were there", which is why charges typically result in criminal liability. They drugs were physically on you, and unless you have evidence that someone planted them, it is unlikely you could establish reasonable doubt.

But in cases of electronic material for networked devices, there is most certainly an affirmative defense to counter actual possession and constructive possession. Computer devices are hacked all the time, and network & device logs exist. For example, if a prosecutor agrees to the fact that a defendant had no knowledge of the material, a judge would toss the case and a jury would not convict you. The law is not meant to pedantically convict you of non-crimes.

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#276

Post by voidnullnil » Sat, Aug 07, 2021, 5:00 PM UTC

skinkestek wrote: post #270

voidnullnil wrote: post #266

Earlier quoted context omitted.

> You can be against this kind of thing from Apple, but as a result more CSAM will be undetected. You cannot claim to be making "the real reasonable analysis" and write this. So much for "you're all geeks stuck on technical details". Quite the contrary: I'm sick of bogus software pretending to solve problems for me, while the quality of tech has exponential degraded over the last 20 years (often due to trying to solv…

Mostly good... > big boy white but why the totally uncalled racism and sexism here? It does nothing to strengthen your argument and it is just dumb.

I am truly sorry for your loss in that your brain is implemented using regex.

I meant "white collar big boys", but I did not bother to edit as I'm writing.

The guy above is claiming everyone who is against apple's yet-another-bogus-TPM-style-snakeoil is a little geek who does not understand anything outside their little tunnel.

Also now that I re-read his comment:

> Edit: After digging in, HN commentary is missing the most relevant details about this particular implementation. iCloud image checking compares to known CSAM image hashes - this means effectively zero false positive rate.

False: it's a perceptual hash. Ignoring the fact that if for some reason you choose to let people host stuff in your icloud account (perhaps as a neat hack), which may be out of terms of service, but certainly not worth 20 years of jail: perceptual hashes have false positives, and can confuse images that appear harmless but were crafted to look like $badimg. But you don't have to be technical to understand that having your devices police you is bad, you just have to not be blinded by politics and boogeyman your state has sold you.

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#277

Post by voidnullnil » Sat, Aug 07, 2021, 5:15 PM UTC

whoknowswhat11 wrote: post #17

voidnullnil wrote: post #12

When I was 13 and my parent made me use a content filter on the web I bypassed it and watched porn and they never found out. On the other hand: Why would I ever want a piece of tech that reports me to the police (even if for legitimate reasons). EDIT: >Anonymous helplines and guidance exist for adults with at-risk thoughts and behaviour [ https://www.apple.com/v/child-safety/a/images/guidance-img__... ] LOL NVM I TRI…

Then apple will lose market share and correct their ways. Conversely, what I've seen does put this top of list as a parent. Will notify me if my child is sending nudes. Will notify me if someone is sending porn to my child. Will notify police if known child porn is on the device. When folks talk about competition - part of this MUST include the USERS preferences (not as currently done the focus of what I see as large…

> I don't want child porn on my systems. Be very happy if apple helps keep it off them.

Apple is scanning your own _personal_ storage for illegal content. It wouldn't be there in the first place, unless you put it there.

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#278

Post by wizzwizz4 » Sat, Aug 07, 2021, 6:13 PM UTC

mulmen wrote: post #264

wizzwizz4 wrote: post #262

Earlier quoted context omitted.

Given corporate personhood, destroying the companies could be considered corporate murder. (I admit it's a very stretched reading.)

I agree that is a favorable reading. But it is especially a stretch in the context of January 6th which involved violence directed at people. And the rest of the paragraph laments that future action must be nonviolent.

So, in fact, it's not advocating the violence.

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#279

Post by 8note » Sat, Aug 07, 2021, 6:18 PM UTC

zepto wrote: post #72

whartung wrote: post #57

Earlier quoted context omitted.

Similarly, many people want the TLAs to be able to go after $2T companies as well.

Do they? What behavior do they want them to go after?

I imagine the IRS for taxes

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#280

Post by mulmen » Sat, Aug 07, 2021, 6:38 PM UTC

wizzwizz4 wrote: post #278

mulmen wrote: post #264

Earlier quoted context omitted.

I agree that is a favorable reading. But it is especially a stretch in the context of January 6th which involved violence directed at people. And the rest of the paragraph laments that future action must be nonviolent.

So, in fact, it's not advocating the violence.

It’s hard to say. Words like “I’d like” and “unfortunately” make me think it’s a desire for violence but an acknowledgement that it’s not worthwhile. Like I said, I’m trying to find the favorable reading. It seems hyperbolic at best.