This can also be used to make Apple's system useless, no? If enough (millions?) of people were to, say, go to a web site and save generated gray-blobs to their phones, it would create enough false-positives to kill this system, right? Maybe game it and have everyone convert their various profile pics to these images.
Hash collision in Apple NeuralHash model
271–280 of 725 posts
Re: Hash collision in Apple NeuralHash model
#272Earlier quoted context omitted.
>> there is no law who requires them to "scan" on device. There is. Apple must comply with warrant requests. If they have a system for scanning files on customer devices they must, if presented with a warrant, allow police access to that system. We can quibble about jurisdictions and constitutional protections, but if the FBI shows up with a federal warrant demanding that Apple remotely scan Sandworm101's phone for a…
That's an incomplete statement. Currently, they must comply with warranty requests by scanning if they have the ability to scan . If they have no such ability (say, because they designed their phones from a privacy-first perspective), the law makes no requirement that they create such a capability. And that's what pisses people off about this.
[deleted]
Re: Hash collision in Apple NeuralHash model
#273Earlier quoted context omitted.
Cross-posting from another thread [1]: 1. Obtain known CSAM that is likely in the database and generate its NeuralHash. 2. Use an image-scaling attack [2] together with adversarial collisions to generate a perturbed image such that its NeuralHash is in the database and its image derivative looks like CSAM. A difference compared to server-side CSAM detection could be that they verify the entire image, and not just the…
Right. So, sending actual CSAM would also work as an attack, but would be detected by the victim and could be corrected (delete images). But a conceivable novel avenue of attack would be to find an image that: 1. Does not look like CSAM to the innocent victim in the original 2. Does match known CSAM by NeuralHash 3. Does look like CSAM in the "visual derivative" reviewed by Apple, as you highlight.
> So, sending actual CSAM would also work as an attack, but would be detected by the victim and could be corrected (delete images).
What if they are placed on the iDevice covertly? Say you want to remove politician X from office. If you got the money or influence you could use a tool like Pegasus (or whatever else there is out there that we don't know of) to place actual CSAM images on their iDevice. Preferably with an older timestamp so that it doesn't appear as the newest image on their timeline. iCloud notices unsynced images and syncs them while performing the CSAM check, it comes back positive with human review (cause it was actual CSAM) and voilà X got the FBI knocking on their door. Even if X can somehow later proof innocence by this time they'll likely have been removed from office over the allegations.
Thinking about it now it's probably even easier: Messaging apps like WhatsApp allow you to save received images directly to camera roll which then auto-syncs with iCloud (if enabled). So you can just blast 30+ (or whatever the requirement was) CSAM images to your victim while they are asleep and by the time they check their phone in the morning the images will already have been processed and an investigation started.
Re: Hash collision in Apple NeuralHash model
#274Earlier quoted context omitted.
Do you really care what they "admit"? I thought you were worried about innocent people being framed. Obviously if a way to frame people gets widespread, Apple will stop it. They don't want that publicity.
You clearly have me confused with someone else, as I never mentioned anything about innocent people being framed. With Apple, nothing is "obvious".
The comment above that I responded to seemed to talk about that. But in any case, I for one don't care what Apple admits.
But I am certain they will not want all the bad publicity that would come if the system was widely abused, if you worry about that. That much is actually "obvious", they are not stupid.
Re: Hash collision in Apple NeuralHash model
#275Apple should be embarrassed for itself. The “one in trillion hash” was cracked in a week. Everyone associated with that should be fired.
Re: Hash collision in Apple NeuralHash model
#276Earlier quoted context omitted.
Don't feel bad at all. I dumped macOS entirely from production workflow. I cannot work on computer knowing that something is "scanning" me and I am glad that my "paranoid" feeling stopped me to upgrade all office macs. Billionaires at (Apple) don't give a flying f*ck about users privacy. It is all vertical integration in the name of world domination. How removed from reality they are. This is week after Pegasus/NSO a…
> and there is no law who requires them to "scan" on device Yet . The demands by "concerned parents" (aka fronts for secret services, puritans/other religious fundamentalists and law-and-order hardliners) to "do something against child porn" have grown ever more strong and insane over the last years. (And you can bet that what is used on CSAM will immediately be used to go after legal pornography, sex work, drug enfo…
> "and what is suspiciously lacking in Apple's response: what are they going to do when they are compelled to extend the CSAM scanner by law in the US, India, China and/or EU?"
from https://daringfireball.net/linked/2021/08/09/apple-csam-faq - "we will not accede to any government’s request to expand it."
From https://www.msn.com/en-us/news/technology/craig-federighi-sa... - "“We ship the same software in China with the same database we ship in America, as we ship in Europe. If someone were to come to Apple [with a request to scan for data beyond CSAM], Apple would say no. But let’s say you aren’t confident. You don’t want to just rely on Apple saying no. You want to be sure that Apple couldn’t get away with it if we said yes,” he told the Journal. “There are multiple levels of auditability, and so we’re making sure that you don’t have to trust any one entity, or even any one country, as far as what images are part of this process.”" - Craig Federighi
Re: Hash collision in Apple NeuralHash model
#277Earlier quoted context omitted.
"Verified by a human" - and that human will be an overworked, underpaid, overseas subcontractor who may well have an incentive to mash the "Confirm match" button from time to time to improve his performance.
I find it very hard to believe that Apple will put literal future of iPhone (just imagine the amount of bad press if one false negative comes out of review process and is reported to the authorities) to "overworked, underpaid overseas subcontractor". Apple is greedy, ruthless, tone-deaf machine, but I don't think they're stupid.
That’s already how app reviews work.
Re: Hash collision in Apple NeuralHash model
#278Earlier quoted context omitted.
>> useful for areas where Apple really doesn't want to even look at the actual material Correct. It has plausible deniability built in. Apple is unable to verify that the images the government are looking for are actually CSAM. They could be political. They could be protest images. They could be Winnie the Pooh. Apple can plead ignorance as it blindly scans for whatever the requesting government asks it to scan for.…
Except that Apple will review the photos once you've matched 30 of them, so it's still not possible for the government to misuse it.
In China, iCloud is run by the government.
Re: Hash collision in Apple NeuralHash model
#279Earlier quoted context omitted.
Linux, dumbphone, DSLR. This was the final straw on a planned exit to be honest.
That's interesting, I may just do the same thing as well - the camera is the largest thing of why I want to be on an phone. I may just go LineageOS.
If you have a Pixel, or are willing to buy one, CalyxOS [1] may be worth a look. It's a privacy-focused ROM that still integrates microG, so it's compatible with most apps (unlike the other popular privacy-focused ROM GrapheneOS [2] which doesn't support microG).
The big advantage to CalyxOS or GrapheneOS versus Lineage is they support re-locking the bootloader, which means that verified boot still works - important if your phone gets swiped.
Re: Hash collision in Apple NeuralHash model
#280Earlier quoted context omitted.
So no need of hash collisions then. One can simply directly send the child porn images to that person via WhatsApp and send her to jail.
But then you're searching and finding child porn images to send.
I'd be surprised if there won't be a darknet service that does exactly that (send CSAM via $POPULAR_MESSENGER) the moment Apple activates scanning.