Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apple’s new abuse prevention system: an antritust/competition point of view

blog.quintarelli.it

Post reply on HN
Print view View original

131–140 of 318 posts

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#131

Post by mthoms »

artificial wrote: post #63
zepto wrote: post #41

Earlier quoted context omitted.

> The governments that twisted Apple's arm to get this look-see into everyone's devices will roast them in the court of public opinion Nothing about this technology gives governments a ‘look see’ into everyone’s devices.

The government controls the hash list.

No. A registered charity called The National Center for Missing and Exploited Children controls the hash list.

Yes, they are partly government funded, but I highly doubt they'd let their mission be compromised by allowing the government to inject non-CP hashes. Doing so would compromise all the work they've performed over the last four decades.

These people are (rightfully) very passionate about their work and can't be easily paid off.

There are many, many valid concerns about this Apple initiative. But, "The government" injecting non CP images into the CP databases is not one of them.

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#132

Post by Conlectus »

DSingularity wrote: post #119
paul_f wrote: post #93

Earlier quoted context omitted.

This seems naive, there are always false positives

No. You can design a system where the FPR is essentially zero. Even if shitty md5 is used.

Perceptual hashes are not exact hashes, otherwise they would be useless for this task; you would just mirror the image or change 1 pixel.

They are instead fuzzy classifiers, and thus have non-zero error rates.

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#133

Post by gonehome »

JoshTko wrote: post #130
gonehome wrote: post #58

Earlier quoted context omitted.

> "As mentioned in the article, this does absolutely nothing towards protecting children other than directing all but the biggest idiots towards platforms that can't be linked to them, which I'd imagine, they already are." I suspect you're more wrong than you think about this. People share large volumes of CSAM through lots of different services - I knew someone who worked on the problem at Linked In (!). HN likes to…

Is CSAM a specific list of images that does not change, or does a government body actively control this list? Is there anything that would technically prevent the addition of a specific image of an enemy of the state? Hypothetical question, how will Apple respond when the FBI asks Apple to scan for images they know are also on the phone of the latest domestic terrorist? And how likely do you think the FBI will try to…

I don't know - I think these are the good questions and the things I would want to know in more detail. I think this kind of thing is where the risk lies.

I think this is why getting the details right matter because if people are arguing about unrelated nonsense it's harder to focus on the actual risks represented by the questions you're asking.

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#135

Post by mikewarot »

mthoms wrote: post #113
lamontcg wrote: post #60

> But when a backdoor is installed, the backdoor exists and history teaches that it’s only a matter of time before it’s also used by the bad guys and authoritarian regimes. Problem is that this scanning is necessarily fuzzy and there is going to be a false positive rate to it. And the way that you'll find out that you've tripped a false positive is that the SWAT team will knock your door down and kill your dog (at a…

> Problem is that this scanning is necessarily fuzzy and there is going to be a false positive rate to it. And the way that you'll find out that you've tripped a false positive is that the SWAT team will knock your door down and kill your dog (at a minimum). Not true. Hash matches are to be human reviewed. So no, people won't get "swatted" accidentally as you allege. The other concerns people have been voicing are ce…

>Hash matches are to be human reviewed.

Until it proves too expensive, then a different AI system will do it instead. I have zero faith that it'll be a fully competent, well trained, well rested, well paid person will actually be doing these reviews in the long run.

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#136

Post by gonehome »

skavi wrote: post #127
gonehome wrote: post #117

Earlier quoted context omitted.

Then that's a different photo and will have a different hash.

is there really no fuzziness to it? If not, can’t this be defeated by simply reencoding the image?

I think it has gotten more sophisticated to detect cropped images and small changes now: https://inhope.org/EN/articles/what-is-image-hashing

The example is somewhat contrived.

If a 'friend' takes your phone and has access to it and then uses it to take images of CSAM similar enough to the original image that it triggers the hash match and does this enough times to go over Apple's threshold to flag the account after these images are uploaded to icloud without the original phone owner noticing then yes it might cause a match.

At that point the match is probably a good thing (and not really a false positive anyway) - since it may lead back to the friend (that has the illegal material).

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#137

Post by 2pEXgD0fZ5cF »

gonehome wrote: post #58
least wrote: post #28

There is no such thing as a trustworthy third party and even trusting yourself is questionable at the best of times. We are constantly balancing a bunch of different considerations with regards to the way that we compute, purchase devices, and utilize services. Security and privacy are of course important, and Apple to date has had a fairly good (if shallow) track record in this regard, at least in the United States.…

> "As mentioned in the article, this does absolutely nothing towards protecting children other than directing all but the biggest idiots towards platforms that can't be linked to them, which I'd imagine, they already are." I suspect you're more wrong than you think about this. People share large volumes of CSAM through lots of different services - I knew someone who worked on the problem at Linked In (!). HN likes to…

One of the prerequisites to "discuss the tradeoffs" of this ordeal is the trust that the people behind it, and those in the position to push things further are actually interested in keeping the scope limited.

There are diminishing returns to using the same old excuse again and again. I'd say most people are just tired of the whole "Just think of the children!" into "Ah we got this system in place why not use it against too" followed by "It's the law in China/Turkey/Russia/wherever, can't just ignore it (and thus not help putting reporters, critics and other people into prison)" combo.

What you are saying is basically another rendition of "look the problem of child abuse exists and this could help so it is worth discussing", which is also a variant of the same.

> iCloud image checking compares to known CSAM image hashes - this means effectively zero false positive rate.

Actually we recently had news where an Apple tried to help cover up their errors [1], the system was supposed to be safe™, doesn't mean the people having control over it can't make mistakes, or worse.

What details are being missed here, exactly? Ultimately it is trivial to expand the hashes to compare to, isn't it? What does it matter that they use CSAM for now? It doesn't remove the involvement of humans in controlling the system, so false positive rate will never be "effectively zero", and it can easily be expanded.

We are left with the same two arguments as always:

- Think about the children

- Just trust the company, they use technology™, no you aren't allowed to check. Yeah they can easily abuse it, but we don't know for sure!

I wouldn't say HN is ignoring details, many people are just tired of the same old loop, there is no reason to put trust into these endeavors and it is reasonable to doubt even the motives.

[1]: https://news.ycombinator.com/item?id=27878333

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#139

Post by ctvo »

DSingularity wrote: post #125
ctvo wrote: post #123

I don't think this issue is widespread enough and harmful enough to society to risk the privacy of all Apple users. Full stop. I haven't been particularly impressed with Apple's security record[1] lately, and I don't trust them to not mess this up. 1 - https://bhavukjain.com/blog/2020/05/30/zeroday-signin-with-a...

Why do you think this?

I can't find a widely accepted statistic but these numbers should illustrate the point:

- A 2016 study by the Center for Court Innovation found that between 8,900 and 10,500 children, ages 13 to 17, are commercially exploited each year in the United States. (Center for Court Innovation, 2016) https://www.courtinnovation.org/sites/default/files/document...

- The annual number of persons prosecuted for commercial sexual exploitation of children (CSEC) cases filed in U.S. district court nearly doubled between 2004 and 2013, increasing from 1,405 to 2,776 cases. https://www.ojp.gov/sites/g/files/xyckuh241/files/archives/p...

This is a niche crime from everything I've seen.

Apple, if it were truly interested in the net good of children, could have picked something that impacts more of them (nutrition? early childhood education?), didn't introduce new vulnerability / abuse surface area, and was less politicized.

Re: Apple’s new abuse prevention system: an antritust/competition point of view

#140

Post by nomel »

Conlectus wrote: post #132
DSingularity wrote: post #119

Earlier quoted context omitted.

No. You can design a system where the FPR is essentially zero. Even if shitty md5 is used.

Perceptual hashes are not exact hashes, otherwise they would be useless for this task; you would just mirror the image or change 1 pixel. They are instead fuzzy classifiers, and thus have non-zero error rates.

But this trigger requires many of these false flags to exceed the threshold (most likely why it exists). I imagine the "one in a trillion" numbers they claim for false flagging rate are probably cemented in reality, and make it trivial for human review.
Post reply on HN