Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A catalog of naturally occurring images whose Apple NeuralHash is identical

github.com

Post reply on HN
Print view View original

51–60 of 304 posts

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#51

Post by willis936 »

xadhominemx wrote: post #48
willis936 wrote: post #42

Earlier quoted context omitted.

2 collisions out of a million images. I'm not sure how big the CSAM database is but if it's a tens of thousands and there are millions of photos uploaded a day then Apple could have a problem on their hands. This is all extrapolating from a study that doesn't use photos representative of what people actually upload. I would suspect when most photos being uploaded are of humans the actual collision rate will be much h…

They don’t take any action unless you have 30 matches in the database, which will not happen by chance.

I would agree if we assume that this very deterministic system behaved randomly.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#52

Post by read_if_gay_ »

toxik wrote: post #30
read_if_gay_ wrote: post #19

Earlier quoted context omitted.

The “actual argument” against the system that this provides is that Apple lied about the likelihood of hash collisions. Therefore, why trust any of their other claims?

I find this an unconvincing argument as well, you're saying that because Apple made a false claim, any claim may be valid. This is obviously not the case, what they did was to, albeit likely knowingly, calculate the hash collision probability /if each bit is a coin flip/, which comes out to pow(2, -k) for k bits. It's tiny. Of course, each bit is /not/ an independent coin flip under the NeuralHash function. So again…

Fine, let’s say they didn’t lie, they made misleading claims.

Still. Why trust them after that?

If a company can make my own smartphone report me to the police, and they want my business, they better prove I can trust them. Apple has plainly done the opposite.

The whole ordeal is just utterly 1984.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#53

Post by [deleted] »

simion314 wrote: post #35
ithkuil wrote: post #24

Earlier quoted context omitted.

Let's not forget what the alternative is: this is about images that are uploaded on icloud anyway. The alternative is to upload the image in clear (or with ane encryption key that apple controls), and let apple run the CSAM filter on their servers. Apple now has the ability to encrypt the images before sending them to icloud, with a private key you own. Except that some percentage of images that match the CSAM finger…

But if Apple really cares about children why they did not done this scans in iCloud like all the others? Did not care as much as Google or Facebook? Seems to me like Apple does not care at all and seems more like a dev with big ego wanted to add neural hashes to his CV but if you can explain how Apple cared for children all this years but only now are doing something I really want to see the explanation

[deleted]

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#54

Post by roody15 »

collaborative wrote: post #15
toxik wrote: post #10

Earlier quoted context omitted.

You (or at least Apple's customers) trust in and rely on Apple's proprietary software to do its job all the time. How is this different? I find this argument very weak.

Tell this to the victims of Pegasus. If anyone were able to get their hands on the "secret backend system" we wouldn't be talking about spy games, we'd be talking about people's lives being ruined

Apple still has not patched the security exploit in iMessage used by Pegasus. Apple has released two ios security updates since the Pegasus revelations but still has not patched it most widely used exploit… hmmmm.

Now apple is getting a local client side scanning tool ready. Interesting timing.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#55

Post by grlass »

koolhaas wrote: post #46
roody15 wrote: post #33

Apple has yet to make a valid reason for implementing client side CSAM scanning. According to Apple only images that will be uploaded to iCloud will be scanned. If this is the case there is zero reason to scan locally and you can just scan the uploaded image once it is on the server. Apple has not implemented E2E nor has it released a statement indicating this will be implemented in the future.

Presumably, it’s done this way so they can say computers other than your personal device do not scan photos and “look” at decrypted and potentially innocent photos. And technically the original image is never decrypted in iCloud by Apple - if 30 images are flagged they are then able to decrypt the CSAM scan meta data which contains resized thumbnails, for confirmation. In summary, I’m guessing they tried to invent a…

calling resized thumbnails metadata is a bit of a stretch imo.

Surely that's just the data, but resized?

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#56

Post by grlass »

willis936 wrote: post #42
slownews45 wrote: post #28

"This is a false-positive rate of 2 in 2 trillion image pairs (1,431,168^2)" That is not bad. As a tool to filter down what apple human reviewers need to look at this is pretty good. Ultimately these images will make it to a human reviewer who can make a call as they would in any flagging system. Could a backend server side system do a more precise hash (96 bits is not a ton) prior to human review?

2 collisions out of a million images. I'm not sure how big the CSAM database is but if it's a tens of thousands and there are millions of photos uploaded a day then Apple could have a problem on their hands. This is all extrapolating from a study that doesn't use photos representative of what people actually upload. I would suspect when most photos being uploaded are of humans the actual collision rate will be much h…

This blogpost [1] by security researcher Sarah Jamie Lewis suggests that the false positive rate could be very high:

[1] https://pseudorandom.resistant.tech/obfuscated_apples.html

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#57

Post by yeldarb »

saithound wrote: post #25
toxik wrote: post #3

Sigh, for the last time, it doesn't actually matter if the NeuralHash is identical. You need multiple images matching, and then the images are compared by another system on Apple's end, which you don't know anything about. The system is specifically designed so that colliding images does not pose a threat to the user. NeuralHash and the CSAM scanning is grotesque, but please, criticize it for what it is, not some bul…

Discussing the preimage attack on NeuralHash is not technical ignorance. Dismissing the preimage attack as irrelevant is. 0. Most importantly: the existence of a preimage attack makes Apple's system completely useless for its original purpose. The NeuralHash collider allows the producers and distributors of CSAM material to ensure that nearly all of the next generation of CSAM will suffer from hash collisions with pe…

> allows the producers and distributors of CSAM material to ensure that nearly all of the next generation of CSAM will suffer from hash collisions with perfectly innocent images

That’s a really interesting attack vector I hadn’t seen mentioned previously.

Most people are talking about the potential for adversarial images to be sent to users. If they were instead injected into the database itself (either by poisoning real CSAM or social engineering) that would have far wider ramifications.

I wonder what the most widely-saved pornographic images are across iCloud users.

If actual CSAM were perturbed to match the hash of, say, images from the celebrity nude leak a few years back and added to the database then thousands of users could be sent to “human review”. Since the images are actually explicit how would the human reviewers know not to flag them to authorities?

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#58

Post by zabatuvajdka »

koolhaas wrote: post #46
roody15 wrote: post #33

Apple has yet to make a valid reason for implementing client side CSAM scanning. According to Apple only images that will be uploaded to iCloud will be scanned. If this is the case there is zero reason to scan locally and you can just scan the uploaded image once it is on the server. Apple has not implemented E2E nor has it released a statement indicating this will be implemented in the future.

Presumably, it’s done this way so they can say computers other than your personal device do not scan photos and “look” at decrypted and potentially innocent photos. And technically the original image is never decrypted in iCloud by Apple - if 30 images are flagged they are then able to decrypt the CSAM scan meta data which contains resized thumbnails, for confirmation. In summary, I’m guessing they tried to invent a…

Interesting technical problem/solution. Another benefit is saving on millions of server computations when modern iOS devices have neural chips etc.

I suppose folks who don’t like privacy implications can downgrade to an iPhone 4 and maybe it will not support the feature.

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#59

Post by saithound »

slownews45 wrote: post #28

"This is a false-positive rate of 2 in 2 trillion image pairs (1,431,168^2)" That is not bad. As a tool to filter down what apple human reviewers need to look at this is pretty good. Ultimately these images will make it to a human reviewer who can make a call as they would in any flagging system. Could a backend server side system do a more precise hash (96 bits is not a ton) prior to human review?

Keep in mind that Apple's claimed false positive rate (one in a trillion chance of an account being flagged innocently), and the collision rate determined by Dwyer in the blog post linked from the repo [2], are both derived without making any adversarial assumptions. Given that NeuralHash collider and similar tools already exist, the practical false positive rate is now expected to be much much higher.

Imagine that you play a game of craps against an online casino. The casino throws a virtual six-sided die, secretly generated using Microsoft Excel's random number generator. Your job is to predict the result. If you manage to predict the result 100 times in a row, you win and the casino will pay you $1000000000000 (one trillion dollars). If you ever fail to predict the result of a throw, the game is over, you lose and you pay the casino $1 (one dollar).

A casino that makes no adversarial assumptions about the clientele could argue as follows: the probability that you accidentally win the game is much less than one in one trillion, so this game is very safe, and the House Edge is excellent [3]. But this number is very misleading: it's based on naive assumptions that are completely meaningless in an adversarial context. Some of the clientele will cheat. If your adversary has a decent knowledge of mathematics at the high school level, the serial correlation in Excel's generator comes into play [4], and the relevant probability is no longer less than 1/1000000000000. In fact, the probability that the client will win is closer to 1/216 instead! When faced with a class of adversarial math majors, a casino that offers this game will promptly go bankrupt. With Apple's CSAM detection, you get to be that casino.

(reposted based on my comment on last week's thread [1])

[1] https://news.ycombinator.com/item?id=28236102

[2] https://blog.roboflow.com/neuralhash-collision/

[3] https://wizardofodds.com/gambling/house-edge/

[4] How to crack a linear congruential generator? http://www.reteam.org/papers/e59.pdf

Re: A catalog of naturally occurring images whose Apple NeuralHash is identical

#60

Post by roody15 »

koolhaas wrote: post #46
roody15 wrote: post #33

Apple has yet to make a valid reason for implementing client side CSAM scanning. According to Apple only images that will be uploaded to iCloud will be scanned. If this is the case there is zero reason to scan locally and you can just scan the uploaded image once it is on the server. Apple has not implemented E2E nor has it released a statement indicating this will be implemented in the future.

Presumably, it’s done this way so they can say computers other than your personal device do not scan photos and “look” at decrypted and potentially innocent photos. And technically the original image is never decrypted in iCloud by Apple - if 30 images are flagged they are then able to decrypt the CSAM scan meta data which contains resized thumbnails, for confirmation. In summary, I’m guessing they tried to invent a…

Apple frequently decrypts icloud data including photos based on a valid warrant. This new local scanning method does not stop apple from complying and decrypting images like they have for years.

https://www.apple.com/legal/privacy/law-enforcement-guidelin...

(Note: I have worked with law enforcement in the past specifically on a case involving Apple and two iCloud accounts. You submit a PDF of the valid warrant to Apple. Apple sends two emails one with the iCloud data encrypted. A second email with the decryption key.)

Post reply on HN