Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

One Bad Apple

hackerfactor.com

Post reply on HN
Print view View original

551–557 of 557 posts

Re: One Bad Apple

#551

Post by logicalmonster »

hackerfactor1 wrote: post #115
tgsovlerkhgsel wrote: post #7

NCMEC has essentially shows that they have zero regard for privacy and called all privacy activists "screeching voices of the minority". At the same time, they're at the center point of a highly opaque, entrenched (often legally mandated) censorhip infrastructure that can and will get accounts shut down irrecoverably and possibly people's homes raided, on questionable data: In one of the previous discussions, I've se…

> I'm surprised, and honestly disappointed, that the author seems to still play nice, instead of releasing the whitepaper. I'm the author. I've worked with different parts of NCMEC for years. (I built the initial FotoForensics service in a few days. Before I wrote the first line of code, I was in phone calls with NCMEC about my reporting requirements.) Over time, this relationship grew. Some years, I was in face-to-f…

> About this time, someone usually mocks "it's always about the kids, think about the kids." To those critics: They have not seen the scope of this problem or the long term impact. There is nearly a 1-to-1 relationship between people who deal in CP and people who abuse children. And they rarely victimize just one child. Nearly 1 in 10 children in the US will be sexually abused before the age of 18.

Does CP being available create victims? I'd say that virtually everybody who suddenly saw CP would not have the inclination to abuse a child. I don't believe that availability of CP is the causal factor to child abuse.

But putting aside that and other extremely important slippery slope arguments for a minute about this issue: have you considered that this project may create economic incentives that are inverse of the ostensible goal of protecting more children from becoming victims?

Consider the following. If it becomes en vogue for cloud data operators to scan their customers' photos for known illegal CP images, then the economic incentives created heavily promote the creation of new, custom CP that isn't present in any database. Like many well-intentioned activists, there's a possibility that you may be contributing more to the problem you care about than actually solving it.

Re: One Bad Apple

#552

Post by DSMan195276 »

skinkestek wrote: post #547
DSMan195276 wrote: post #513

Earlier quoted context omitted.

> I'd say we get police or health care to talk to people who think perfectly normal images are sexual in nature, but until we get laws changed at least then keep us safe. Personally I don't find anecdotes convincing compared to the very real amount of CSAM (and actual child abuse) we already know exists and circulates in the wild, but I do get your point. That said personally I don't think changing the laws would rea…

> Personally I don't find anecdotes convincing compared to the very real amount of CSAM (and actual child abuse) we already know exists First: This is not hearsay or anecdotal evidence, this is multiple innocent real people getting their lives trashed to some degree before getting aquitted. > I don't think a random Walmart employee is up-to-date on the legal definitions of CSAM, they're going to potentially report it…

> First: This is not hearsay or anecdotal evidence, this is multiple innocent real people getting their lives trashed to some degree before getting aquitted.

"multiple" is still anecdotal, unless we have actual numbers on the issue. The question is how many of these cases actually happen vs. the number of times these types of investigations actually reveal something bad. Unless you never want kids saved from abuse there has to be some acceptable number of investigations that eventually get dropped.

> Remember the job of the police is more to keep law abiding citizens safe than to lock up offenders.

Maybe that should be their purpose, but in reality they're law enforcement, their job has nothing to do with keeping people safe. The SCOTUS has confirmed as much that the police have no duty to protect people, only to enforce the law. However I think we agree that's pretty problematic...

> Making innocent peoples lives less safe for a marginally bigger chance to catch small fry (i.e. not producers), does it matter?

I would point out that the children in this situation are law abiding citizens as well, and they also deserve protection. Whether their lives were made more or less safe in this situation is debatable, but the decision was made with their safety in mind. For the few cases of a mistake being made like the one you presented I could easily find similar cases where the kids were taken away and then it was found they were actually being abused. That's also why I pointed out your examples are only anecdotes, the big question is whether this is a one-off or a wider trend.

If reducing the police's ability to investigate these potential crimes would actually result in harm to more children, then you're really not achieving your goal of keeping people safer.

> The problem here and elsewhere is that police many places doesn't have a good track record of throwing it out. Once you've been dragged through court for the most heinous crimes you don't get your life completely back.

Now this I agree with. The "not having a good record of throwing it out" I'm a little iffy on but generally agree, but I definitely agree that public knowledge of being investigating for such a thing is damaging even if it turns out your innocent, which isn't right. I can't really say I have much of a solution for that in a situation like this though, I don't think there's much of a way to not-publicly take the kids away - and maybe that should have a higher threshold, but I really don't know, as I mentioned earlier we'd really need to look at the numbers to know that. For cases that don't involve a public component like that though I think there should be a lot more anonymity involved.

Re: One Bad Apple

#553

Post by stjohnswarts »

hackerfactor1 wrote: post #115
tgsovlerkhgsel wrote: post #7

NCMEC has essentially shows that they have zero regard for privacy and called all privacy activists "screeching voices of the minority". At the same time, they're at the center point of a highly opaque, entrenched (often legally mandated) censorhip infrastructure that can and will get accounts shut down irrecoverably and possibly people's homes raided, on questionable data: In one of the previous discussions, I've se…

> I'm surprised, and honestly disappointed, that the author seems to still play nice, instead of releasing the whitepaper. I'm the author. I've worked with different parts of NCMEC for years. (I built the initial FotoForensics service in a few days. Before I wrote the first line of code, I was in phone calls with NCMEC about my reporting requirements.) Over time, this relationship grew. Some years, I was in face-to-f…

I'm glad that you at least were transparent about the shortcomings of these types of systems. Maybe some academics in forensic image analysis can take up the torch and shed light on how much of a failure this whole system is, especially when they're putting spyware on our personal devices. Anyway, I like your writings, keep up the good fight.

Re: One Bad Apple

#554

Post by a-dub »

upbeat_general wrote: post #549
a-dub wrote: post #546

Earlier quoted context omitted.

i don't think they would necessarily advertise a plan to attempt e2e this early. what if it proved to be infeasible? reversing course after an announcement like that would be a massive black eye. have you ever seen a system like this that would be capable of flagging accounts for hosting bad material with a tiny false positive rate while being capable of e2e encrypting the material at rest like this one? i haven't, a…

...this early? no e2e on iCloud has been a major issue for years. It's not like they're beta testing (the underlying principle/structure that is) and again, I haven't seen a system that flags bad material that is also e2e because the whole premise is flawed.

and obviously a contentious issue...

they probably already do server side scanning, and probably regularly find stuff.

assuming that was true (which it very well could be) it would be insanely irresponsible to roll out e2e at scale without something like this...

there are probably hundreds of people or more at apple who can legitimately access the contents of a user account. e2e with inbound scanning would completely ameliorate that.

Re: One Bad Apple

#555

Post by microChasm »

nullc wrote: post #153
ziml77 wrote: post #61

Earlier quoted context omitted.

They could have done all that without telling you. And as long as the traffic was combined with normal traffic no one would ever notice (and in this case it would end up mixed with normal traffic since it only applies to images being uploaded to iCloud, so communication with Apples servers would be expected). What it looks like to me is that Apple is planning on releasing end-to-end encryption for iCloud. But they kn…

> They could have done all that without telling you. But in that case it would much more likely be a crime, it would certainly cost them a tremendous amount of good will. Your personal computing device is a trusted agent. You cannot use the internet without it, and esp. in lockdown you likely can't realistically live your life without use of the internet. You share with it your most private information, more so even…

Eh, I think it is simply the fact that Apple doesn’t want to be associated with individuals violating their terms of service in an unlawful way.

This is a way to root them out and report them to law enforcement.

Re: One Bad Apple

#556

Post by microChasm »

nullc wrote: post #143
kemayo wrote: post #100

> To reiterate: scanning your device is not a privacy risk, but copying files from your device without any notice is definitely a privacy issue. I think the article is wrong about this. Or, right-but-situationally-irrelevant. As far as I can tell from Apple's statements, they're doing this only to photos which are being uploaded to iCloud Photos. So, any photo this is happening to is one that you've already asked App…

> There is a chance of false positives, so the human review step seems necessary... You misunderstand the purpose of the human review by Apple. The human review is not due to false positives: The system is designed to have an extremely low rate of hits where the entry isn't in the database and the review invades your privacy regardless of who does it. The human review exists to legitimize an otherwise unlawful search…

This is the part I am very concerned about. This is definitely a violation of 4th Amendment rights because images are viewed by humans not on the device. What happened to just on device scanning for them?

Re: One Bad Apple

#557

Post by microChasm »

microChasm wrote: post #556
nullc wrote: post #143

Earlier quoted context omitted.

> There is a chance of false positives, so the human review step seems necessary... You misunderstand the purpose of the human review by Apple. The human review is not due to false positives: The system is designed to have an extremely low rate of hits where the entry isn't in the database and the review invades your privacy regardless of who does it. The human review exists to legitimize an otherwise unlawful search…

This is the part I am very concerned about. This is definitely a violation of 4th Amendment rights because images are viewed by humans not on the device. What happened to just on device scanning for them?

There is one thing to be concerned about individuals violating terms of service and scanning on the device to identify and refer to law enforcement. It’s a WHOLE other thing to have humans somehow review images that are not in a device.
Post reply on HN