Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ImageNet contains naturally occurring Apple NeuralHash collisions

blog.roboflow.com

Post reply on HN
Print view View original

351–360 of 530 posts

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#351

Post by c7DJTLrn »

wobbly_bush wrote: post #345
gsibble wrote: post #329

Earlier quoted context omitted.

The feeling is mutual. The devices we own are now being used to actively police us.

The parent comment might have misunderstood what the government asked for. It asked for a feature to "report spam" by end users.

Maybe I did. But what difference does it make? There's plenty of other instances where Apple has reportedly been bullied into action or inaction (being dissuaded from implementing E2EE for iCloud is one example). I've really just reached a breaking point and I'm sorry if logic does not apply.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#352

Post by gremloni »

t0mas88 wrote: post #332
c7DJTLrn wrote: post #322

Earlier quoted context omitted.

I'm so done. I'm sorry to dump a pointless rant like this on HN but... what the hell is going on these days? Nobody seriously seems to care about legitimate privacy concerns anymore. If I were in a position of power, like being CEO, CTO, or even just an engineer on the team at Apple that implemented this, I'd do EVERYTHING to make sure that my power is in check and that I'm not pushing a fundamentally harmful technol…

I'm as surprised as you are that a giant like Apple doesn't just tell them "go ahead, ban iPhones, see how popular they'll become" to someone as powerless as the government of India. It would be a huge free publicity campaign for them in the rest of the world while the public in India would either put pressure on their government or buy iPhones via import websites. For additional fun, strike a deal with the #2 non go…

“Powerless” is not how I would refer to the Indian government.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#353

Post by biztos »

tshaddox wrote: post #158
stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

If the CCP says “put this arbitrary software into your next iPhone software update or we will halt all iPhone sales in China,” what do you think Apple is going to do? Isn’t the answer to both questions the same?

It's a fair question, but I think the answer is no: the questions are not the same.

As much as Apple wants access to the Chinese market, it would (presumably) draw a line at some point where it would (presumably) have to choose between that market and the US market, if only because the latter is both its legal domicile and the source of most of its talent.

Version A: CCP wants to exploit the hash database, there are lots of ways to do that, bullying Apple is one, any other way gives Apple a "we are looking into it" excuse. "We must comply with local laws, but we will not change our software bla bla."

Version B: CCP wants to exploit iOS, only way to do it is to bully Apple, this forces Apple's hand and very possibly Apple moves production (not just sales) out of China because they no longer trust they will be offered "plausible deniability."

I'm sure there are lots of reasons for that absurd cash reserve, but my best guess is it's to cover the eventuality of B. above; Apple talking about that publicly would be tricky.

https://www.cnbc.com/2020/07/30/apple-q3-cash-hoard-heres-ho...

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#354

Post by FridayoLeary »

t0mas88 wrote: post #332
c7DJTLrn wrote: post #322

Earlier quoted context omitted.

I'm so done. I'm sorry to dump a pointless rant like this on HN but... what the hell is going on these days? Nobody seriously seems to care about legitimate privacy concerns anymore. If I were in a position of power, like being CEO, CTO, or even just an engineer on the team at Apple that implemented this, I'd do EVERYTHING to make sure that my power is in check and that I'm not pushing a fundamentally harmful technol…

I'm as surprised as you are that a giant like Apple doesn't just tell them "go ahead, ban iPhones, see how popular they'll become" to someone as powerless as the government of India. It would be a huge free publicity campaign for them in the rest of the world while the public in India would either put pressure on their government or buy iPhones via import websites. For additional fun, strike a deal with the #2 non go…

Too political. It would just scare consumers away.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#355

Post by drdeca »

tshaddox wrote: post #319
noptd wrote: post #285

Earlier quoted context omitted.

So we should simply accept systems with a high potential for abuse because of the possibility that something bad is already being done?

What has more potential for abuse than the fact that Apple can push any software they want to iPhones at any time?

Can they force install updates? I think it’s been over a year since I updated my iPhone’s os.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#356

Post by KallDrexx »

biesnecker wrote: post #41
endisneigh wrote: post #39

Earlier quoted context omitted.

Microsoft, Facebook and Google. More depending on what services you use

They're not actively scanning your phone, they're actively scanning files you send them.

That's not actually answering the question in the GP about why this is different.

Photos people send me to my Android are automatically sent through 3rd parties, either through MMS, Facebook messenger, Google Photos, or One Drive. Photos arriving on my device are almost guaranteed to be uploaded to both OneDrive and Google Photos based on how defaults of Android phones are setup.

So someone could already send hash collisions my way (purposely or inadvertently) and authoritarian governments already have access in their respective clouds (at least China does).

And yet, there are not stories of people being falsely accused of child porn due to PhotoDNA hash collisions.

Why does "on device for apple devices only" change the calculus.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#357

Post by rsj_hn »

c7DJTLrn wrote: post #322
captn3m0 wrote: post #115

Earlier quoted context omitted.

Related, the Indian Government (Telecom Department) bullied Apple into building an iOS feature for reporting phone calls and SMS by threatening to stop iPhone sales in India. Apple complied. https://indianexpress.com/article/technology/mobile-tabs/app...

I'm so done. I'm sorry to dump a pointless rant like this on HN but... what the hell is going on these days? Nobody seriously seems to care about legitimate privacy concerns anymore. If I were in a position of power, like being CEO, CTO, or even just an engineer on the team at Apple that implemented this, I'd do EVERYTHING to make sure that my power is in check and that I'm not pushing a fundamentally harmful technol…

What is going on is that reality is slapping some techno-utopians in the face and they are shocked, shocked, that governments are more powerful that businesses.

That's not at all what the lefty geeks learned by reading Chomsky or what the righty geeks learned by reading Heinlein.

All along these people thought algorithms and protocols (e.g. bitcoin and TCP/IP) would somehow be a powerful force that would cause governments to fall on their knees and let people evade government control. After all, it's distributed! You can't stop it!

Well, that was all very foolish, because they mistook government uninterest in something for the equivalent of government being powerless to control it, and when governments did start taking an interest in something, it turns out that protocols and algorithms are no defense against the realities of political power. It is to the field of politics, and not the field of technology, that one must turn in order to increase collective freedoms. Individual freedom can be increased by obtaining money or making lots of friends, but collective freedom cannot be increased this way, it can only be increased by organizing and influencing government.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#358

Post by ChrisKnott »

saithound wrote: post #343
ChrisKnott wrote: post #311

Earlier quoted context omitted.

Can you explain how these theoretical political memes hash-match to an image in the NCMEC database, and then also pass the visual check? > "No, this misses the point completely. You cannot easily trigger any automated systems merely by taking photos of 17.9 year olds and sending them to people." Did I say "taking"? I am talking about sending (theoretical) actual images from the NCMEC database. This is functionally id…

Yes, I can. This is just one possible strategy: there are many others, where different things are done, and where things are done in a different order. You use the collider [1] and one of the many scaling attacks ([2] [3] [4], just the ones linked in this thread) to create an image that matches the hash of a reasonably fresh CSAM image currently circulating on the Internet, and resizes to some legal sexual or violent…

Ok yeah, I do agree this scaling attack potentially makes this feasible, if it essentially allows you to present a completely different image to the reviewer as to the user. Has anyone done this yet? i.e. an image that NeuralHashes to a target hash, and also scale-attacks to a target image, but looks completely different.

(Perhaps I misunderstood your original post, but this seems to be a completely different scenario to the one you originally described with reference to the three thumbnails)

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#359

Post by FabHK »

t0mas88 wrote: post #332
c7DJTLrn wrote: post #322

Earlier quoted context omitted.

I'm so done. I'm sorry to dump a pointless rant like this on HN but... what the hell is going on these days? Nobody seriously seems to care about legitimate privacy concerns anymore. If I were in a position of power, like being CEO, CTO, or even just an engineer on the team at Apple that implemented this, I'd do EVERYTHING to make sure that my power is in check and that I'm not pushing a fundamentally harmful technol…

I'm as surprised as you are that a giant like Apple doesn't just tell them "go ahead, ban iPhones, see how popular they'll become" to someone as powerless as the government of India. It would be a huge free publicity campaign for them in the rest of the world while the public in India would either put pressure on their government or buy iPhones via import websites. For additional fun, strike a deal with the #2 non go…

How much fun would that be for customers if India then decided to confiscate every iPhone it encounters within India (maybe excepting tourists, but maybe not)?

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#360

Post by pfranz »

slg wrote: post #48
bawolff wrote: post #9

Earlier quoted context omitted.

ImageNet is a very well-known data set. Are we sure apple didn't test on it when designing this algorithm?

>This is a false-positive rate of 2 in 2 trillion image pairs (1,431,168^2). Assuming the NCMEC database has more than 20,000 images, this represents a slightly higher rate than Apple had previously reported. But, assuming there are less than a million images in the dataset, it's probably in the right ballpark. Apple reported a pretty similar collision rate so maybe they did.

The only number I've heard from Apple is, "the likelihood that the system would incorrectly identify any given account is less than one in one trillion per year."[1] Which I read as enough false hits to flag an account that year (some interview said that threshold was around 30). That depends on the average number of new photos uploaded to iCloud, the size of the NCMEC database, the threshold for flagging the account, and the error rate of the match. Without knowing most of those numbers it's hard to gauge how close it is.

https://www.apple.com/child-safety/pdf/Expanded_Protections_...

Post reply on HN