Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ImageNet contains naturally occurring Apple NeuralHash collisions

blog.roboflow.com

Post reply on HN
Print view View original

331–340 of 530 posts

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#331

Post by domador »

c7DJTLrn wrote: post #322
captn3m0 wrote: post #115

Earlier quoted context omitted.

Related, the Indian Government (Telecom Department) bullied Apple into building an iOS feature for reporting phone calls and SMS by threatening to stop iPhone sales in India. Apple complied. https://indianexpress.com/article/technology/mobile-tabs/app...

I'm so done. I'm sorry to dump a pointless rant like this on HN but... what the hell is going on these days? Nobody seriously seems to care about legitimate privacy concerns anymore. If I were in a position of power, like being CEO, CTO, or even just an engineer on the team at Apple that implemented this, I'd do EVERYTHING to make sure that my power is in check and that I'm not pushing a fundamentally harmful technol…

There are many of us that DO care! Unfortunately, even though we are many, we are still a small minority among the general population, or probably even among software developers.

Convenience and fashion tend to trump security and principles for most people. (Oftentimes, I'm one of those people as well, though I try not to be. It's exhausting to be an activist 100% of the time. But let's keep at it!)

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#332

Post by t0mas88 »

c7DJTLrn wrote: post #322
captn3m0 wrote: post #115

Earlier quoted context omitted.

Related, the Indian Government (Telecom Department) bullied Apple into building an iOS feature for reporting phone calls and SMS by threatening to stop iPhone sales in India. Apple complied. https://indianexpress.com/article/technology/mobile-tabs/app...

I'm so done. I'm sorry to dump a pointless rant like this on HN but... what the hell is going on these days? Nobody seriously seems to care about legitimate privacy concerns anymore. If I were in a position of power, like being CEO, CTO, or even just an engineer on the team at Apple that implemented this, I'd do EVERYTHING to make sure that my power is in check and that I'm not pushing a fundamentally harmful technol…

I'm as surprised as you are that a giant like Apple doesn't just tell them "go ahead, ban iPhones, see how popular they'll become" to someone as powerless as the government of India. It would be a huge free publicity campaign for them in the rest of the world while the public in India would either put pressure on their government or buy iPhones via import websites.

For additional fun, strike a deal with the #2 non government owned carrier in whichever country you do this to. Offer the iPhone at a special rate for a few months. They would kill the government telco while selling record numbers of phones with free publicity. And at the same time scare any other government into not trying this kind of stunt with Apple ever again.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#333

Post by dan-robertson »

yeldarb wrote: post #166
dan-robertson wrote: post #157

It didn’t seem to take long for the weights for Apple’s network to be discovered. And I suppose they must send the banned hashes to the client for checking too. So I expect that list will be discovered and published soon too (unless they have some way to keep them secret?) I think one important question is: how reversible is Apple’s perceptual hash? For example, my understanding of Microsoft’s PhotoDNA is that their…

It doesn't seem like it; check out the adversarially constructed images here. They don't look anything like the original despite perfectly matching the NeuralHash: https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX/issue...

Right obviously the hashing isn’t going to be injective and therefore there are lots of silly images that hash to a given value. The question is whether it is possible to efficiently find plausible images with a given hash.

Think more like a “deep dream” than these adversarial attacks.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#334

Post by spideymans »

nomel wrote: post #236
tgsovlerkhgsel wrote: post #141

> Apple now has over 1.5 billion users so we are talking about a large pool of users at stake which increases the likelihood of even a low probability event manifesting This is an extremely good point. If the whole system, end to end, after all safeguards (e.g. human reviewers which can also make mistakes) has a one-in-a-billion chance to ruin a user's life, then statistically, we can expect 1-2 users to have their l…

I think you misunderstand the reporting process. If the threshold is passed, Apple reports the images to NCMEC for review. NCMEC reports it to authorities. So, this would require the failure of three organizations.

It would also require the failure of both the prosecutor and the judiciary to recognize the images as non-CSAM.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#335

Post by dan-robertson »

fortenforge wrote: post #178
dan-robertson wrote: post #157

It didn’t seem to take long for the weights for Apple’s network to be discovered. And I suppose they must send the banned hashes to the client for checking too. So I expect that list will be discovered and published soon too (unless they have some way to keep them secret?) I think one important question is: how reversible is Apple’s perceptual hash? For example, my understanding of Microsoft’s PhotoDNA is that their…

The hashes sent to the client are cryptographically blinded, making it impossible for the client to determine the original CSAM hashes.

This is the information I was missing, thanks.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#336

Post by Youden »

robertoandred wrote: post #266
Youden wrote: post #107

Earlier quoted context omitted.

> It doesn’t matter if there are collisions if the two images don’t actually look the same. Is that really true? My understanding is that the manual reviewers at Apple only see some kind of low-resolution proxy, not the full-resolution image. I'd also be shocked if the human reviewers were shown the original, actually CP image, to compare to. Given that, it's not necessary to produce an actual visual match, it's just…

Why would you save all of the almost-CSAM pics the attacker sends you to your photo library?

Manually, perhaps because the attacker crafts the messages to make that desirable.

But a bigger concern is automatically. WhatsApp for example can automatically save all received photos to your Camera Roll, which is of course automatically backed up to iCloud for many people. So an attacker could potentially just send you a bundle of 40 images of however many and your phone automatically sends it to Apple.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#337

Post by t0mas88 »

dkdk8283 wrote: post #313
numbsafari wrote: post #310

Earlier quoted context omitted.

What makes us think that that isn’t exactly what this is already?

This is already happening to phones. The baseband blobs are proprietary and most devices permit DMA. Nobody really knows what the blobs do. They likely have paved the way for Stingrays and other devices.

In Android yes, but Apple has an advantage here that they control the whole device.

If Apple really wanted to they could secure the iPhone. Just offer a 2x higher bug bounty than any government and things like stingray would not work.

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#338

Post by int_19h »

azinman2 wrote: post #249
sillysaurusx wrote: post #240

Earlier quoted context omitted.

It’s blinded in the cryptographic sense. It’s a specific term. I would go into detail, but . Suffice to say, unless you provide proof, I am reasonably confident there’s no way to verify the hash db doesn’t contain extra hashes other than the CSAM hashes provided by the US government. But I’ve been wrong many times before.

Well first of all, it's not provided by the US government. It's a non-profit, and Apple has already said they're going to look for another db from another nation and only included hashes that are the union of the two to prevent exactly this kind of attack. If what you mean by blinded is that you don't know what the source image is for the hash, that's true. Otherwise Apple would just be putting a database of child po…

It's a non-profit that is effectively maintained by the US government (funding, special legal status, interaction with government agencies etc).

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#339

Post by rovr138 »

alfalfasprout wrote: post #2

There's been a lot of focus on the likelihood of collisions and whether someone could upload eg; an image with a matching hash to your device to "set you up", etc. But what's still extremely concerning is that there is still no guarantee that the hash list used can't be coopted for another purpose (eg; politically insensitive content).

On top of that, what happens if a court/government orders them to give them all the current data about people with matches, regardless of the 30 matches.

They can't say if it is or not a match so they have to go after the individuals. Is that enough evidence for a warrant?

Someone in the court thinks it's true and can't prosecute?, oh, it got leaked.

--

Not every country has the same protections about innocent until proven guilty. And even then, we've seen cases in the US where someone has been held in jail indefinitely until they provide a password,

- https://arstechnica.com/tech-policy/2016/04/child-porn-suspe...

- https://nakedsecurity.sophos.com/2016/04/28/suspect-who-wont...

Re: ImageNet contains naturally occurring Apple NeuralHash collisions

#340

Post by Peaches4Rent »

captn3m0 wrote: post #115
stickfigure wrote: post #19

> it's not obvious how we can trust that a rogue actor (like a foreign government) couldn't add non-CSAM hashes to the list to root out human rights advocates or political rivals. Apple has tried to mitigate this by requiring two countries to agree to add a file to the list, but the process for this seems opaque and ripe for abuse. If the CCP says "put these hashes in your database or we will halt all iPhone sales in…

Related, the Indian Government (Telecom Department) bullied Apple into building an iOS feature for reporting phone calls and SMS by threatening to stop iPhone sales in India. Apple complied. https://indianexpress.com/article/technology/mobile-tabs/app...

Misinformation. Thats for reporting spam.

It's a feature that's could be done by anyone using sms anyway.

I guess apple had to add an easy way

Post reply on HN