One Bad Apple • Hacker News Board

Re: One Bad Apple

#141

Post by zug_zug » Mon, Aug 09, 2021, 1:44 AM UTC

hackerfactor1 wrote: post #115

tgsovlerkhgsel wrote: post #7

NCMEC has essentially shows that they have zero regard for privacy and called all privacy activists "screeching voices of the minority". At the same time, they're at the center point of a highly opaque, entrenched (often legally mandated) censorhip infrastructure that can and will get accounts shut down irrecoverably and possibly people's homes raided, on questionable data: In one of the previous discussions, I've se…

> I'm surprised, and honestly disappointed, that the author seems to still play nice, instead of releasing the whitepaper. I'm the author. I've worked with different parts of NCMEC for years. (I built the initial FotoForensics service in a few days. Before I wrote the first line of code, I was in phone calls with NCMEC about my reporting requirements.) Over time, this relationship grew. Some years, I was in face-to-f…

>> There is nearly a 1-to-1 relationship between people who deal in CP and people who abuse children

Can you give a source for this please? Also by "deal in" do you mean create or view?

Re: One Bad Apple

#142

Post by XorNot » Mon, Aug 09, 2021, 1:45 AM UTC

jolux wrote: post #103

IronWolve wrote: post #62

There was parents arrested over bath time and playing in the yard sprinklers, photos being processed at photo mats, will the same thing happen by apple mistakenly reporting parents? When I worked telecom, we had md5sum database to check for this type of content. If you emailed/sms/uploaded a file with the same md5sum, your account was flagged and sent to legal to confirm it. Also if a police was involved, the account…

> There was parents arrested over bath time and playing in the yard sprinklers, photos being processed at photo mats, will the same thing happen by apple mistakenly reporting parents? No, because they’re not identifying content, they’re matching it against a set of already-known CSAM that NCMEC maintains. As you go on to say, telecoms and other companies already do this. Apple just advanced the state of the art when…

It's a lossy hash match though. If it wasn't, then subtly re-encoding the image would hide it. So they're definitely going to be mistakingly matching images.

Re: One Bad Apple

#143

Post by nullc » Mon, Aug 09, 2021, 1:46 AM UTC

kemayo wrote: post #100

> To reiterate: scanning your device is not a privacy risk, but copying files from your device without any notice is definitely a privacy issue. I think the article is wrong about this. Or, right-but-situationally-irrelevant. As far as I can tell from Apple's statements, they're doing this only to photos which are being uploaded to iCloud Photos. So, any photo this is happening to is one that you've already asked App…

> There is a chance of false positives, so the human review step seems necessary...

You misunderstand the purpose of the human review by Apple.

The human review is not due to false positives: The system is designed to have an extremely low rate of hits where the entry isn't in the database and the review invades your privacy regardless of who does it.

The human review exists to legitimize an otherwise unlawful search via a loophole.

The US Government (directly or through effective agencies like NCMEC) is barred from searching or inspecting your private communications without a warrant.

Apple, by virtue of your contractual relationship with them, is free to do so-- so long as they are not coerced to do so by the government. When Apple reviews your communications and finds what they believe to be child porn they're then required to report it and because the government is merely repeating a search that apple already (legally) performed, no warrant is required.

So, Apple "reviews" the hits because, per the courts, if they just sent automated matches without review that wouldn't be sufficient to avoid the need for a warrant.

The extra review step does not exist to protect your privacy: The review itself deprives you of your privacy. The review step exists to suppress your fourth amendment rights.

Re: One Bad Apple

#144

Post by heavyset_go » Mon, Aug 09, 2021, 1:48 AM UTC

q-rews wrote: post #130

heavyset_go wrote: post #102

As a fan of this blog for longer than I can remember, it's refreshing to hear this particular author's take on this issue, especially considering their background. I'm glad these issues were addressed in a much more elegant way than I would have put them: > Apple's technical whitepaper is overly technical -- and yet doesn't give enough information for someone to confirm the implementation. (I cover this type of paper…

Regarding that rate, I’m no expert but my guess is that it’s the result of math, not actual testing of 1+ trillion images. This sounds like calling bullshit on “You have one trillion chance to win the lottery.”

The author addresses this point:

> Perhaps Apple is basing their "1 in 1 trillion" estimate on the number of bits in their hash? With cryptographic hashes (MD5, SHA1, etc.), we can use the number of bits to identify the likelihood of a collision. If the odds are "1 in 1 trillion", then it means the algorithm has about 40 bits for the hash. However, counting the bit size for a hash does not work with perceptual hashes.

> With perceptual hashes, the real question is how often do those specific attributes appear in a photo. This isn't the same as looking at the number of bits in the hash. (Two different pictures of cars will have different perceptual hashes. Two different pictures of similar dogs taken at similar angles will have similar hashes. And two different pictures of white walls will be almost identical.)

> With AI-driven perceptual hashes, including algorithms like Apple's NeuralHash, you don't even know the attributes so you cannot directly test the likelihood. The only real solution is to test by passing through a large number of visually different images. But as I mentioned, I don't think Apple has access to 1 trillion pictures.

> What is the real error rate? We don't know. Apple doesn't seem to know. And since they don't know, they appear to have just thrown out a really big number. As far as I can tell, Apple's claim of "1 in 1 trillion" is a baseless estimate. In this regard, Apple has provided misleading support for their algorithm and misleading accuracy rates.

Re: One Bad Apple

#145

Post by tux3 » Mon, Aug 09, 2021, 1:48 AM UTC

hackerfactor1 wrote: post #115

tgsovlerkhgsel wrote: post #7

NCMEC has essentially shows that they have zero regard for privacy and called all privacy activists "screeching voices of the minority". At the same time, they're at the center point of a highly opaque, entrenched (often legally mandated) censorhip infrastructure that can and will get accounts shut down irrecoverably and possibly people's homes raided, on questionable data: In one of the previous discussions, I've se…

> I'm surprised, and honestly disappointed, that the author seems to still play nice, instead of releasing the whitepaper. I'm the author. I've worked with different parts of NCMEC for years. (I built the initial FotoForensics service in a few days. Before I wrote the first line of code, I was in phone calls with NCMEC about my reporting requirements.) Over time, this relationship grew. Some years, I was in face-to-f…

I'd lament that using 'thinking of the children' as an excuse for genuine overreach, when that's what it is, puts us in a worse situation.

There is incredible second-order harm in overreach, because the reaction to it hurts the original cause, too.

If you try to overcorrect, people will overcorrect in response.

The sort of zeal that leads to thoughts like "screeching minority", I think shows carelessness and shortsightedness in the face of very important decisions.

I have no informed opinion on Apple's CSAM tool, beyond deferring to the FotoForensics expert.

Re: One Bad Apple

#146

Post by gscott » Mon, Aug 09, 2021, 1:50 AM UTC

Kim Dotcom did this and it lead to Riaa saying it obligated the file sharing service to look for all copyrighted materials. This opens pandoras box for Apple.

Re: One Bad Apple

#147

Post by kbenson » Mon, Aug 09, 2021, 1:51 AM UTC

slg wrote: post #132

hackerfactor1 wrote: post #115

Earlier quoted context omitted.

> I'm surprised, and honestly disappointed, that the author seems to still play nice, instead of releasing the whitepaper. I'm the author. I've worked with different parts of NCMEC for years. (I built the initial FotoForensics service in a few days. Before I wrote the first line of code, I was in phone calls with NCMEC about my reporting requirements.) Over time, this relationship grew. Some years, I was in face-to-f…

>About this time, someone usually mocks "it's always about the kids, think about the kids." To those critics: They have not seen the scope of this problem or the long term impact. There is nearly a 1-to-1 relationship between people who deal in CP and people who abuse children. And they rarely victimize just one child. Nearly 1 in 10 children in the US will be sexually abused before the age of 18. I think we have see…

> Too often the tech community's response is that the intangible concept of privacy is more important than the tangible issue of child abuse.

I think the actual idea, which the nuance is often lost (and let's be honest, some people aren't really aware of and are just jumping on the bandwagon), is that privacy is a right, and erosion of rights is extremely important because it has been shown many times in the past to have far reaching and poorly understood future effects.

It's unfortunate that some would choose to link their opposition to the idea with making the thing it's attempting to combat legal when it's something so abhorrent, but in general I think a push back on any erosion of our rights is a good and useful way for people to dedicate time and effort.

While we shouldn't lose sight that there are real children in danger and that this should help, we also shouldn't lose sight that there is plenty of precedence that this could eventually lead to danger and problems for a great many other people, children included.

Re: One Bad Apple

#148

Post by slg » Mon, Aug 09, 2021, 1:51 AM UTC

zug_zug wrote: post #140

slg wrote: post #132

Earlier quoted context omitted.

>About this time, someone usually mocks "it's always about the kids, think about the kids." To those critics: They have not seen the scope of this problem or the long term impact. There is nearly a 1-to-1 relationship between people who deal in CP and people who abuse children. And they rarely victimize just one child. Nearly 1 in 10 children in the US will be sexually abused before the age of 18. I think we have see…

> Too often the tech community's response is that the intangible concept of privacy is more important than the tangible issue of child abuse. Is it intangible? 18% of the world lives in China alone. That's more people than the "1/10 who are victims of child abuse*", and I'm sure that 18% will only grow as other authoritarian countries get more technologically advanced. I think "Think of the kids" applies very well to…

>Is it intangible? 18% of the world lives in China alone. That's more people than the "1/10 who are victims of child abuse*", and I'm sure that 18% will only grow as other authoritarian countries get more technologically advanced.

You didn't mention any tangible results here. How would this system by Apple make my life worse? Can you answer that without a slippery slope argument?

>I think "Think of the kids" applies very well to the CREATORS of pornography. Per wikipedia, there isn't any conclusive causal relationship between viewing CP and assaulting children.

Why does the causality matter? A correlation is enough that cracking down on this content will result in less abusers on the streets.

>* Per a google search "A Bureau of Justice Statistics report shows 1.6 % (sixteen out of one thousand) of children between the ages of 12-17 were victims of rape/sexual assault" which is a lot less than 10% figure you're citing. Non-sexual abuse wouldn't really have any bearing here, right?

I wasn't the one citing that, but you are also citing an incomplete number since it excludes younger children.

Re: One Bad Apple

#149

Post by tablespoon » Mon, Aug 09, 2021, 1:53 AM UTC

heavyset_go wrote: post #104

Thorrez wrote: post #56

Earlier quoted context omitted.

>I'm surprised, and honestly disappointed, that the author seems to still play nice, instead of releasing the whitepaper. Would it help anything? Apple isn't using PhotoDNA, so proving PhotoDNA is bad would just be met with "we don't use that".

I believe it would because other image systems will probably have to make similar implementations and trade offs that PhotoDNA did.

>> Would it help anything? Apple isn't using PhotoDNA, so proving PhotoDNA is bad would just be met with "we don't use that".

> I believe it would because other image systems will probably have to make similar implementations and trade offs that PhotoDNA did.

Even if that's the case, that's too subtle of a point to be very effective, and you'll have all kinds of PR people (Apple and otherwise) selling the "we don't use that [so those criticisms don't apply to us]" narrative.

Re: One Bad Apple

#150

Post by prawn » Mon, Aug 09, 2021, 1:54 AM UTC

IronWolve wrote: post #62

There was parents arrested over bath time and playing in the yard sprinklers, photos being processed at photo mats, will the same thing happen by apple mistakenly reporting parents? When I worked telecom, we had md5sum database to check for this type of content. If you emailed/sms/uploaded a file with the same md5sum, your account was flagged and sent to legal to confirm it. Also if a police was involved, the account…

You can read a bit more about how it would work here:

https://daringfireball.net/2021/08/apple_child_safety_initia...

(No, not arresting parents over bath time.)