Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ImageNet contains naturally occurring Apple NeuralHash collisions

blog.roboflow.com

Post reply on HN
Print view View original

321–330 of 530 posts

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#321

Post by [deleted] »

mrtranscendence wrote: post #179
saithound wrote: post #163

Keep in mind that Apple's claimed false positive rate (one in a trillion chance of an account being flagged innocently), and the collision rate determined by Dwyer in the article, are both derived without any adversarial assumptions. Given that NeuralHash collider and similar tools already exist, the false positive rate is expected to be much much higher. Imagine that you play a game of craps against an online casino…

Why would anyone bother with such an attack? The end result is that some peon at Apple has to look at the images and mark them as not CSAM. You've cost someone a bit of privacy, but that's it.

[deleted]

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#322

Post by c7DJTLrn »

captn3m0 wrote: post #115
stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

Related, the Indian Government (Telecom Department) bullied Apple into building an iOS feature for reporting phone calls and SMS by threatening to stop iPhone sales in India. Apple complied. https://indianexpress.com/article/technology/mobile-tabs/app...

I'm so done. I'm sorry to dump a pointless rant like this on HN but... what the hell is going on these days? Nobody seriously seems to care about legitimate privacy concerns anymore. If I were in a position of power, like being CEO, CTO, or even just an engineer on the team at Apple that implemented this, I'd do EVERYTHING to make sure that my power is in check and that I'm not pushing a fundamentally harmful technology.

I just feel so lost and powerless these days, I don't know how much longer I can go on when every piece of technology I own is working against me - tools designed to serve a ruling class instead of the consumer. I don't like it one bit.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#323

Post by vmladenov »

giantrobot wrote: post #225
yongjik wrote: post #110

Earlier quoted context omitted.

These scenarios sound rather like "the wrong side of airlock" stories[1]. Why would China go through an elaborate scheme with fake child-porn hashes, when it can already arrest these people on made-up charges, and simply tell Apple to provide the private key for their phones, so that they can read and insert whatever real/fake evidences they want? [1] I'm stealing the expression from this excellent article: https://d…

China or any government adding collisions would be to use Apple's system as a dragnet to find users possessing the offending images. The way it would work is the government in question would submit legitimate CSAM but modified to produce a collision with a government target image. Looking at the raw image (or a derivative) a reviewer at Apple or ICMEC would see a CSAM image. The algorithm would see the anti-governmen…

> So Apple scans Chinese (or whoever) citizens libraries, finds "CSAM" and reports them to ICMEC which then reports them to the government in question.

If Apple finds that a particular hash is notorious for false positives, they can reject it / ask for a better one. And they’re not scanning your library; it’s a filter on upload to iCloud. The FUD surrounding this is getting ridiculous.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#324

Post by gonehome »

stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

The CCP already runs iCloud themselves in country so this is a bit irrelevant. (Though I think this kind of capitulation to authoritarian countries is wrong, personally: https://zalberico.com/essay/2020/06/13/zoom-in-china.html)

This policy really needs to be compared to the status quo (unencrypted on cloud image scanning). When you compare in transit client side hash checks that allow for on cloud encryption and only occur when using the cloud it's hard to see an argument against it that makes sense given the current setup.

The abuse risk is no higher than the status quo and enabling encryption is a net win.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#325

Post by Notanothertoo »

stefan_ wrote: post #282
nomel wrote: post #216

Earlier quoted context omitted.

Why is this question being downvoted? I too would like to know what this attack achieves. From what I see, the end result of false flagging is either someone has CSAM in iCloud and you push them over the threshold that results in reporting and prosecution, or there is no CASM, so the reviewer sees all of the hash collision images, including those that are natural. Is the problem that an attacker can force natural has…

You are one underpaid random guy in India looking at CSAM all day clicking the wrong button away from a raid of your home and the end of your life as you know it.

This, these charges are damning once they are made. Plus the countless legal dollars you are going to have to front and hours spent proving innocence and that's assuming the justice system actually works.. Try explain this to your employer while you start missing deadlines due to court dates.. The police also could easily leverage this to warrant hop. As they have been found doing in the past. I think the bike rider who had nothing to do with a crime and got accused because he was the only one in a broad geo location Warren is all the president of of you need that this will be abused.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#326

Post by short_sells_poo »

mrtranscendence wrote: post #206
minitoar wrote: post #11

Earlier quoted context omitted.

The damage is already done by the time it gets to the point of devices being confiscated.

If you don't actually have CSAM then the person at Apple who visually audits the collision set will not send your info to law enforcement, and nothing will be confiscated. Apple doesn't just take any instance of a collision and send it to the FBI.

Right, because humans are infallible and never abuse their position either.

And we are not talking about some underpaid contractors making life altering decisions.

And we are also not talking about a system where just a false accusation can and will destroy one's life irreparably.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#327

Post by short_sells_poo »

nonbirithm wrote: post #118
WesolyKubeczek wrote: post #106

Earlier quoted context omitted.

You know, it's rather not okay to treat every smartphone user out there as a potential criminal just because they happen to have photos on their devices. At least in an alleged democracy where there's this presumption of innocence thing. Even Pegasus wasn't that much rotten, it at least wasn't indiscriminately installed onto everyone's phone. But what can I say, there wasn't much uproar about piracy tax on blank CD-R…

And it's fine to treat everyone as a potential criminal when we entrust our data on the same company's servers? No matter if on-device scanning makes surveillance easier than ever, the surveillance itself was still a significant possibility up to this point with server-side scanning. Imagine how many petabytes of user data already exist in the cloud.

So in your mind because bad thing X is already happening, it's completely OK for bad thing XY to also start happening?

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#328

Post by kburman »

captn3m0 wrote: post #115
stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

Related, the Indian Government (Telecom Department) bullied Apple into building an iOS feature for reporting phone calls and SMS by threatening to stop iPhone sales in India. Apple complied. https://indianexpress.com/article/technology/mobile-tabs/app...

I'm amazed on how much mis-information is spread. Featured we are talking about here is for reporting spam number which is a done by user not automatically. This is widely available in Android already.

Correct me if I'm wrong but this feature needs an app install from the app store.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#329

Post by gsibble »

c7DJTLrn wrote: post #322
captn3m0 wrote: post #115

Earlier quoted context omitted.

Related, the Indian Government (Telecom Department) bullied Apple into building an iOS feature for reporting phone calls and SMS by threatening to stop iPhone sales in India. Apple complied. https://indianexpress.com/article/technology/mobile-tabs/app...

I'm so done. I'm sorry to dump a pointless rant like this on HN but... what the hell is going on these days? Nobody seriously seems to care about legitimate privacy concerns anymore. If I were in a position of power, like being CEO, CTO, or even just an engineer on the team at Apple that implemented this, I'd do EVERYTHING to make sure that my power is in check and that I'm not pushing a fundamentally harmful technol…

The feeling is mutual. The devices we own are now being used to actively police us.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#330

Post by Someone »

alfalfasprout wrote: post #2

There's been a lot of focus on the likelihood of collisions and whether someone could upload eg; an image with a matching hash to your device to "set you up", etc. But what's still extremely concerning is that there is still no guarantee that the hash list used can't be coopted for another purpose (eg; politically insensitive content).

There wasn’t any guarantee that Apple didn’t have and use such technology before they made this feature public, or even already was running it in the current version of iOS.

If you trusted Apple not to stealthily run such technology before, the question is how much less (if any) you trust them now.

If you didn’t, I don’t think anything changed.

Post reply on HN