Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hash collision in Apple NeuralHash model

github.com

Post reply on HN
Print view View original

641–650 of 725 posts

Re: Hash collision in Apple NeuralHash model

#641

Post by bsql »

tick_tock_tick wrote: post #590
cyanite wrote: post #575

Earlier quoted context omitted.

> Just insert a known CSAM image on target's device. Done. What do you mean “just”? That’s not usually very simple. It needs to go into the actual photo library. Also, you need like 30 of them inserted. > I presume this could be used against a rival political party Yes, but it’s not much different from now, since most cloud photo providers scan for this cloud-side. So that’s more an argument against scanning all toge…

> It needs to go into the actual photo library. iMessage photos received are automatically synced so no. Finding 30 photos take zero time at all on Tor. Hell finding a .onion site that doesn't have CP randomly spammed is harder.....

iMessage photos do not automatically add photos to your photo library. Yes they’re synced between devices but afaik Apple isn’t deploying this hashing technology on iMessages between devices. Only for iCloud photos in the photo library.

Re: Hash collision in Apple NeuralHash model

#642

Post by croutonwagon »

stephenr wrote: post #558
romwell wrote: post #537

Earlier quoted context omitted.

>Also, if Twitter, Google, Microsoft are already deploying CSAM scanning in their services .... why are we not hearing about all the "swatting"? >their services >T H E I R S E R V I C E S Because it's on their SERVICES, not on their user's DEVICES, for one. Also, regardless of swatting, that's why we have an issue with Apple.

It only happens on Apple devices right before the content is uploaded to the service. How is that a meaningful difference for the stated end goals, that can explain the lack of precedent.

I think this is where a disconnect is occurring.

In this specific case yes. That is what is supposed to happen.

But Apple also sets the standard that this is just the beginning, not the end. They say as much on page 3 in bold, differentiated color ink

https://www.apple.com/child-safety/pdf/Expanded_Protections_...

And there’s nothing to stop them from scanning all images on a device. Or scanning all content for keywords or whatever. iCloud being used as a qualifier is a red herring to what this change is capable of.

Maybe someone shooting guns is now unacceptable, kids have been kicked from schools for posting them on Facebook or having them in their rooms on zoom. What if it’s kids shooting guns? There are so many possibilities of how this could be misused, abused or even just an oopsie, sorry I upended your life to solve a problem that is so very rare.

Add to that their messaging has been muddy at best. And it incited a flame war. A big part of that is iCloud is not a single thing. It’s a service, it can sync snd hold iMessages, it can sync backups, or in my case We have shared iCloud albums that we use to share images with family. Others are free to upload and share. In fact that’s our only use of iCloud other than find my. They say iCloud photos as if that’s just a single thing but it’s easy to extrapolate that to images in iMessages, backups etc.

And the non profit that hosts this database is not publicly accountable. They have public employees on their payroll but really they can put whatever they want in that database. They have no accountability or public disclosure requirements.

So even I, when their main page was like 3 articles was a bit perturbed and put off. I’m not going to ditch my iPhone, mainly because it’s work assigned but I have been keeping a keen eye on what’s happening, how it’s happening and will keep an eye out for their chnages they are promising. I’m also going to guess they won’t nearly be as high profile in the future.

Re: Hash collision in Apple NeuralHash model

#643

Post by FabHK »

dannyw wrote: post #476
bitneuker wrote: post #190

Earlier quoted context omitted.

A vulnerability by itself is not that dangerous, but in combination with a sophisticated attack, or another vulnerability can be disastrous. State actors have the resources to exploit a number of unknown bugs in combination with this collision to have Apple's systems flag persons of interest. This, combined with human error during the manual review process might result in someone getting reported. Seeing as twitter (…

No one is going to send gray blobs, they will be finding legal porn (like pussy close ups, tongue pics, whatever) and then disturbing it to trigger a CSAM hit. The low res derivative will match, perhaps even closely, because pussy closeups look similar to an apple employee when its grayscale 64 by 64 pixels (remember: it's illegal for Apple to transmit CSAM, so it must be so visually degraded to the point where it's…

Ok, so you posit an attacker could find/generate 30+ pictures that are

1. accepted by innocent user,

2. flagged as known CSAM by NeuralHash,

2b. also flagged by the second algorithm Apple will run over flagged images server side as known CSAM,

3. apparently CSAM in the "visual derivative".

That strikes me as a rather remote scenario, but worth investigating. Having said that, if it's a 3-letter adversary using Pegasus unhappy with a journalist, couldn't they just put actual CSAM onto the journalist's phone? And couldn't they have done that for many years?

Re: Hash collision in Apple NeuralHash model

#644

Post by yayr »

cyanite wrote: post #639
yayr wrote: post #112

How long did it take now to make the Apple algorithm ultimately useless or even harmful? Apple announcement of neural hashing: 5.8.2021. Generic algorithm to generate a different matching image: 8.8.2021. one script was already released 10 days ago here https://gist.github.com/unrealwill/c480371c3a4bf3abb29856c29...

None of this makes the system useless or harmful. Also, it’s not Apple’s algorithm. The actual hash list Apple will use is not accessible to the device.

If anybody with enough motivation can modify any existing harmless image to have the same neural hash as a "tracked database" image this will create too many false positives. Too many false positives make the algorithm useless.

If someone with even more motivation and the means to put those images onto your device via social engineering, exploits or maybe even features and you become the target of a criminal investigation in any jurisdiction you just happen to be at the moment, this makes the algorithm harmful.

Re: Hash collision in Apple NeuralHash model

#645

Post by spoonjim »

cyanite wrote: post #605
spoonjim wrote: post #598

Any idea why Apple had to be too cute by half and not just scan these files server-side? Or why it doesn't change their plan to do that, given the backlash?

I hope not, since doing the scans like this offers much more privacy for the user. Even when the user is too ignorant (sorry) to realize this. But this will be evident by reading and understanding the technical description.

I'm not concerned with this system -- I'm concerned with the things that governments will now start to force Apple (and any computer vendor) to start scanning for on the client side. Once the client side scanning Rubicon is crossed, countries will want vendors to scan everything (not just iCloud folders) for everything (not just CSAM).

Re: Hash collision in Apple NeuralHash model

#646

Post by floatingatoll »

neolog wrote: post #640
floatingatoll wrote: post #557

Earlier quoted context omitted.

Why would they bother? That's a terrible way to approach it. Just pass legislation requiring in-country datacenters that can be decrypted by thoughtcrime enforcers, like Russia and China are doing. Trying to get this done via a CSAM list that's absurdly closely audited would be a huge waste of time and not provide any significant benefit, and if such a request were ever made public, would likely result in severe poli…

Without the technology deployed, Apple can (and did) say they don't have the ability to break into users' phones. If Apple deploys on-phone scanning, governments can just tell Apple to support a new list. It won't be the NCMEC CSAM list. It will be a "public safety and security" list. I wouldn't rule out underhandedness either. [1] [1] https://www.nytimes.com/2020/07/01/technology/china-uighurs-...

Apple already has technology deployed to perform binary file scans of every file on macOS and iOS, and the ability to at any time release signatures for those scans, that are very difficult for normal users to prevent updates for. They've had that for years, maybe even a decade by now, and so far to date we have seen no abuse of that list.

How is Apple's new CSAM list somehow increasing the chances of Apple going rogue, given that we've all been living with that risk for the past X years?

Re: Hash collision in Apple NeuralHash model

#647

Post by FabHK »

cyanite wrote: post #588
dannyw wrote: post #492

Earlier quoted context omitted.

You are aware that a lot of CSAM are close ups of say pussies for example, and human anatomy can look very similar? I'm not talking about images of rape here. I'm taking about images that you'd see on a regular porn site, of adults and their body parts. You are also aware that CSAM covers anywhere from 0 to 17.99 years of age, and the legal obligation to report exists equally for the whole spectrum? So let's say I do…

> You are aware that a lot of CSAM are close ups of say pussies for example, and human anatomy can look very similar? I doubt images that look quite generic will make it into those hash sets, though.

Nearly impossible to verify, though, by construction.

Re: Hash collision in Apple NeuralHash model

#648

Post by rootsudo »

Sanzig wrote: post #279
rootsudo wrote: post #242

Earlier quoted context omitted.

That's interesting, I may just do the same thing as well - the camera is the largest thing of why I want to be on an phone. I may just go LineageOS.

If you have a Pixel, or are willing to buy one, CalyxOS [1] may be worth a look. It's a privacy-focused ROM that still integrates microG, so it's compatible with most apps (unlike the other popular privacy-focused ROM GrapheneOS [2] which doesn't support microG). The big advantage to CalyxOS or GrapheneOS versus Lineage is they support re-locking the bootloader, which means that verified boot still works - important…

Thank you for the info, I will go buy and experiment about this.

Re: Hash collision in Apple NeuralHash model

#649

Post by neolog »

floatingatoll wrote: post #646
neolog wrote: post #640

Earlier quoted context omitted.

Without the technology deployed, Apple can (and did) say they don't have the ability to break into users' phones. If Apple deploys on-phone scanning, governments can just tell Apple to support a new list. It won't be the NCMEC CSAM list. It will be a "public safety and security" list. I wouldn't rule out underhandedness either. [1] [1] https://www.nytimes.com/2020/07/01/technology/china-uighurs-...

Apple already has technology deployed to perform binary file scans of every file on macOS and iOS, and the ability to at any time release signatures for those scans, that are very difficult for normal users to prevent updates for. They've had that for years, maybe even a decade by now, and so far to date we have seen no abuse of that list. How is Apple's new CSAM list somehow increasing the chances of Apple going rog…

What technology are you referring to as already deployed?

Re: Hash collision in Apple NeuralHash model

#650

Post by icelancer »

shapefrog wrote: post #122
icelancer wrote: post #59

Earlier quoted context omitted.

> Of course, grey noise will never pass for CSAM and will fail that step. Never? You sure that one or more human operators will never make this mistake, dooming someone's life / causing them immense pain?

I can guarantee nobody will see the inside of a courtroom, on charges of possession and distribution of child porn for possessing multiple images of grey noise (unless there is some steganography going on).

> I can guarantee nobody will see the inside of a courtroom

This wasn't the question I asked.

Post reply on HN