Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ImageNet contains naturally occurring Apple NeuralHash collisions

blog.roboflow.com

Post reply on HN
Print view View original

381–390 of 530 posts

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#381

Post by nescioquid »

c7DJTLrn wrote: post #322
captn3m0 wrote: post #115

Earlier quoted context omitted.

Related, the Indian Government (Telecom Department) bullied Apple into building an iOS feature for reporting phone calls and SMS by threatening to stop iPhone sales in India. Apple complied. https://indianexpress.com/article/technology/mobile-tabs/app...

I'm so done. I'm sorry to dump a pointless rant like this on HN but... what the hell is going on these days? Nobody seriously seems to care about legitimate privacy concerns anymore. If I were in a position of power, like being CEO, CTO, or even just an engineer on the team at Apple that implemented this, I'd do EVERYTHING to make sure that my power is in check and that I'm not pushing a fundamentally harmful technol…

> I don't know how much longer I can go on when every piece of technology I own is working against me - tools designed to serve a ruling class instead of the consumer.

This made me think of the Butlerian Jihad in Dune:

"Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them."

I'd say the typical remedy that societies have adopted for these sorts of things is legislation, though regulatory capture[1] is an issue that blocks the way.

[1] https://en.wikipedia.org/wiki/Regulatory_capture

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#382

Post by cheald »

The demonstrated ability to produce collisions at will should be an instant show-stopper for this feature.

If a bad actor can send targets innocent images which hash to CSAM, you essentially have an upgraded secret swatting mechanism that people will use to abuse others.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#383

Post by JunkDNA »

rsj_hn wrote: post #357
c7DJTLrn wrote: post #322

Earlier quoted context omitted.

I'm so done. I'm sorry to dump a pointless rant like this on HN but... what the hell is going on these days? Nobody seriously seems to care about legitimate privacy concerns anymore. If I were in a position of power, like being CEO, CTO, or even just an engineer on the team at Apple that implemented this, I'd do EVERYTHING to make sure that my power is in check and that I'm not pushing a fundamentally harmful technol…

What is going on is that reality is slapping some techno-utopians in the face and they are shocked, shocked, that governments are more powerful that businesses. That's not at all what the lefty geeks learned by reading Chomsky or what the righty geeks learned by reading Heinlein. All along these people thought algorithms and protocols (e.g. bitcoin and TCP/IP) would somehow be a powerful force that would cause govern…

Bingo. I wish I could upvote this comment more. All the geeks get distracted by words like “cloud” or “virtual” and forget that all this stuff we depend on has a physical presence at some point in the real world. That physical presence necessitates humans interacting with other humans. Humans interacting with humans falls squarely in the “things governments poke their noses into” bucket. It’s like the early days of Napster when people were all hot for “peer to peer”, as if that tech was some magic that was going to make record labels and governments throw up their hands over copyrights.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#384

Post by cheald »

mrtranscendence wrote: post #179
saithound wrote: post #163

Keep in mind that Apple's claimed false positive rate (one in a trillion chance of an account being flagged innocently), and the collision rate determined by Dwyer in the article, are both derived without any adversarial assumptions. Given that NeuralHash collider and similar tools already exist, the false positive rate is expected to be much much higher. Imagine that you play a game of craps against an online casino…

Why would anyone bother with such an attack? The end result is that some peon at Apple has to look at the images and mark them as not CSAM. You've cost someone a bit of privacy, but that's it.

Why would anyone bother calling the cops and telling them that someone they don't like is an imminent threat? The end result is that some officer just has to stop by and see that they aren't actually building bombs. You've cost someone a bit of time, but that's it.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#385

Post by FabHK »

dgellow wrote: post #121
breuleux wrote: post #67

Earlier quoted context omitted.

Presumably Apple would be afraid that, say, the EU becomes suspicious, issues a court order to obtain the hashes, notices they cannot audit the CCP hashes, pointedly asks "what is this", becomes absolutely livid that their citizens are spied on by a country that is not them, fines Apple out the wazoo, then extradites whoever is responsible and puts them in prison. I mean, China's not the only player in this. Putting…

I think that you overestimate the EU reaction. Every few years we learn that our Europeans leaders and some citizens have been again spied by foreign powers, such as the US, and absolutely nothing ever happened.

> I think that you overestimate the EU reaction.

Great (depressing) Twitter account: Is EU Concerned? @ISEUConcerned Very, deeply, strongly, seriously, gravely, extremely, unprecedentedly

https://twitter.com/ISEUConcerned

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#386

Post by dweekly »

rsj_hn wrote: post #357
c7DJTLrn wrote: post #322

Earlier quoted context omitted.

I'm so done. I'm sorry to dump a pointless rant like this on HN but... what the hell is going on these days? Nobody seriously seems to care about legitimate privacy concerns anymore. If I were in a position of power, like being CEO, CTO, or even just an engineer on the team at Apple that implemented this, I'd do EVERYTHING to make sure that my power is in check and that I'm not pushing a fundamentally harmful technol…

What is going on is that reality is slapping some techno-utopians in the face and they are shocked, shocked, that governments are more powerful that businesses. That's not at all what the lefty geeks learned by reading Chomsky or what the righty geeks learned by reading Heinlein. All along these people thought algorithms and protocols (e.g. bitcoin and TCP/IP) would somehow be a powerful force that would cause govern…

(Gestures at the community of people who smugly use the word "fiat" as commensurate with obsolete)

Not sure everyone has gotten the memo yet.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#387

Post by JohnJamesRambo »

stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

Tank Man from Tiananmen Square image comes to mind.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#388

Post by InTheArena »

endisneigh wrote: post #8

It doesn’t matter if there are collisions if the two images don’t actually look the same. Do people honestly believe a single CSAM flag from an “innocent” image is going to result in someone going to prison in America? PhotoDNA has existed for over a decade doing the same thing with no instances that I have heard of. If some corrupt government wants to get you they don’t need this. They can just unilaterally say you’…

A honest question I have is if PhotoDNA results where ever validated? Is there a chance that collisions have been occurring with no one looking to see if actual CSAM was in use?

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#389

Post by andrei_says_ »

c7DJTLrn wrote: post #322
captn3m0 wrote: post #115

Earlier quoted context omitted.

Related, the Indian Government (Telecom Department) bullied Apple into building an iOS feature for reporting phone calls and SMS by threatening to stop iPhone sales in India. Apple complied. https://indianexpress.com/article/technology/mobile-tabs/app...

I'm so done. I'm sorry to dump a pointless rant like this on HN but... what the hell is going on these days? Nobody seriously seems to care about legitimate privacy concerns anymore. If I were in a position of power, like being CEO, CTO, or even just an engineer on the team at Apple that implemented this, I'd do EVERYTHING to make sure that my power is in check and that I'm not pushing a fundamentally harmful technol…

My only conclusion is that Apple is not arguing in good faith. That CSAM prevention is an excuse.

This makes me very sad.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#390

Post by AussieWog93 »

tshaddox wrote: post #257
giantrobot wrote: post #228

Earlier quoted context omitted.

> they could literally just tell Apple to issue a software update that streams all desired private data to Chinese government servers. Uh...this already happened. [0][1] [0] https://www.macrumors.com/2021/05/17/apple-security-compromi... [1] https://support.apple.com/en-us/HT208351

Not quite. Those are still ostensibly servers located in China but not directly controlled by the government (edit: apparently the hosting company is owned by Guizhou provincial government) . But yes, this is precisely my point. Any slippery slope argument about Apple software on iPhones is equivalent to any conceivable slippery slope argument about Apple software on iPhones. If you're making one of these arguments,…

> ostensibly servers located in China but not directly controlled by the government

"ostensibly" is the key word there. If the datacenter is physically located in China, then there's a CCP official on the board of the company that controls it.

Post reply on HN