The Problem with Perceptual Hashes • Hacker News Board

Re: The Problem with Perceptual Hashes

#411

Post by nullc » Sun, Aug 08, 2021, 1:23 AM UTC

spacedcowboy wrote: post #408

nullc wrote: post #393

Earlier quoted context omitted.

> all cloud companies in the US are required to inspect photos for CSAM) This is extremely disingenuous. If their devices uploaded content with end to end encryption there would be no matches for CSAM. If they were required to search your materials generally, then they would be effectively deputized-- acting on behalf of the government-- and your forth amendment protection against unlawful search would be would exten…

This is not disingenuous - it's a statement of the reality within which we live. You can claim all you like that there is no coercion taking place, and that e2e would solve all ills, but it doesn't change the facts that: - All cloud providers scan for it. Facebook, Google, Amazon, Apple, Imgur ... There's a list of 144 companies at NCMEC. There must be a damn good reason for that consensus... - Because they scan for…

> You can claim all you like that there is no coercion taking place

It's not just myself claiming it, it is google claiming it under oath. If they were perjuring themselves and their scanning was, in fact, coerced by the government it would not change improve the situation: In that case by scanning in response to government coercion they would be aiding an unlawful violation of their user's fourth amendments and covering it it up.

If you'd like to sustain an argument that there is a large conspiracy of tech companies along with the government to violate the public's constitutional rights on a massive scale and lying in court to cover for it-- well I wouldn't be that shocked. But if that's true then it their complicity is a VASTLY greater ethical failure.

I think, however, we should greatly penalize the prospects of a vast conspiracy to violate the public's constitutional rights. It would be far from a new thing for there to be large number of commercial entities violating the civil rights of the public without any government coercion to do so.

> My gut feeling is that the people bemoaning this as if the end-times were here will still all (for reasonable definitions of "all") be using iCloud in a few months time, and having their photos scanned

Well not me. I don't use any of those privacy invading services, and I hope you won't either!

And there are plenty of data storage providers that have users data encrypted by default.

Re: The Problem with Perceptual Hashes

#412

Post by iratewizard » Sun, Aug 08, 2021, 1:48 AM UTC

jefftk wrote: post #409

iratewizard wrote: post #407

Earlier quoted context omitted.

English is a pragmatic language, meaning word order is flexible. In my first sentence, the 'even' can go in both places while retaining the same meaning. Good luck with your ESL tests as well.

> In my first sentence, the 'even' can go in both places while retaining the same meaning. I really don't think it can. Looking through examples of people using that construction [1] they are all being used to emphasize the speaker's lack of something, without any implication that if someone were to have one it would be the speaker. Do you see any examples of someone using it your way? > Good luck with your ESL tests…

I can't even begin to explain how sad that is for you.

Re: The Problem with Perceptual Hashes

#413

Post by chucklenorris » Sun, Aug 08, 2021, 3:06 AM UTC

So, if there's code on the device that's computing these hashes then it can be extracted. Afterwards it should be possible to add changes to a inocent picture to make it produce a target hash. Getting a hash should pe possible too, just find a known pedo image and run the extracted algorithm. It's only a matter of time until someone makes this

Re: The Problem with Perceptual Hashes

#414

Post by chucklenorris » Sun, Aug 08, 2021, 3:09 AM UTC

This technology is a godsend for the government to catch wistleblowers before they're able to leak information. You wouldn't even hear about those poor souls.

Re: The Problem with Perceptual Hashes

#415

Post by TeeMassive » Sun, Aug 08, 2021, 5:32 AM UTC

amannm wrote: post #370

TeeMassive wrote: post #243

Earlier quoted context omitted.

You don't even need hacking for this to be abused by malevolent actors. A wife in a bad marriage could simply take nude pictures of their child to falsely accuse her husband. This tech is just ripe for all kind of abuses.

That picture wouldn't already be in the CSAM database...

There's no point for this tech then. There's no way that they will no try to expand to detect the presence of porn and children being both present at the same time.

Re: The Problem with Perceptual Hashes

#416

Post by jbuhbjlnjbn » Sun, Aug 08, 2021, 9:03 AM UTC

thaumasiotes wrote: post #148

ezoe wrote: post #110

The problem of hash or NN based matching is, the authority can avoid explaining the mismatch. Suppose the authority want to false-arrest you. They prepare a hash that matches to an innocent image they knew the target has in his Apple product. They hand that hash to the Apple, claiming it's a hash from a child abuse image and demand privacy-invasive searching for the greater good. Then, Apple report you have a file th…

> like the dogs which conveniently bark at police's secret hand sign This isn't necessary; the state of the art is for drug dogs to alert 100% of the time. They're graded on whether they ever miss drugs. It's easy to never miss.

To be more precise, if the dog always barks and only missing positives are counted, it is inevitable to never miss. An obvious number cheat.

Re: The Problem with Perceptual Hashes

#417

Post by visarga » Sun, Aug 08, 2021, 9:13 AM UTC

Klonoar wrote: post #222

visarga wrote: post #203

Earlier quoted context omitted.

It's not the same because before the hashing was done in the cloud, but now the model is accessible locally, you just need to take pictures. This means it's easier to hack. If someone discovers a way to reliably generate adversarial images they can send such images to someone else to iSWAT them.

If your definition of "hack" is "get bob to accept bad file", no, this model is not easier - it's just different . You could literally piggyback on the directories that Macs use to sync to iCloud Drive, get an image in there, and then it gets scanned by iCloud. This is not some new theoretical attack - and in fact, this would be the "hack" for the new one as well since it requires iCloud sync to trigger anyway .

I was referring to a "white box" adversarial attack, meaning you can try variations of the input to see how the outcome changes, and then construct an input that will fool the system. That's only possible if you have access to the perceptual hashing model, which you do now since it runs locally.

Of course generating an adversarial image is not the final step, the hacker still need to place it in the victim's account somehow, but it's easier now because the image looks legit.

Re: The Problem with Perceptual Hashes

#418

Post by afiori » Sun, Aug 08, 2021, 9:36 AM UTC

jokoon wrote: post #276

nicce wrote: post #246

Earlier quoted context omitted.

Corruption. Lack of evidence on some other cases. Personal revenge. Who knows, but list is big.

Ok but but what ends?

performance metrics and careers exists in law enforcement too.

eg:

https://listverse.com/2016/11/06/10-undercover-cops-who-went...

Re: The Problem with Perceptual Hashes

#419

Post by jefftk » Sun, Aug 08, 2021, 11:56 AM UTC

iratewizard wrote: post #412

jefftk wrote: post #409

Earlier quoted context omitted.

> In my first sentence, the 'even' can go in both places while retaining the same meaning. I really don't think it can. Looking through examples of people using that construction [1] they are all being used to emphasize the speaker's lack of something, without any implication that if someone were to have one it would be the speaker. Do you see any examples of someone using it your way? > Good luck with your ESL tests…

I can't even begin to explain how sad that is for you.

I'd interpret that as "this is incredibly sad for your", as in, it's so sad it's hard to even begin to explain it. Do you interpret it as "if anyone could explain how sad this is it would be me, and I can't"?

(It would be helpful if you could link to an example in context, where we can see that the speaker is using it your way)

Re: The Problem with Perceptual Hashes

#420

Post by asimpletune » Sun, Aug 08, 2021, 12:58 PM UTC

jeromegv wrote: post #330

asimpletune wrote: post #105

Earlier quoted context omitted.

I have been a big Apple fan ever since my first computer. This is the first time I legitimately thought I need to start thinking about something else. It’s kind of sad.

Genuinely curious, why? This scanning was already happening server-side in your iCloud photos, just like Google Photos, etc. Now they are removing it from server-side to client-side (which still require this photo to be hosted in iCloud) What changed, really?

The problem for me is my device “informing” on me changes the fundamental nature of my device. Also the opaqueness of the whole system is concerning, along with the potential for false positives. And lastly is the “if this then what’s next?” And that is definitely a road I don’t want to go down on.

For me it’s sad because I have literally always stood by them and they make amazing hardware and software. However at the end of the day I’d rather have the nature of my device be one where it is under my control, than all the wonderful apple tech.

I just don’t know what to use instead.