141–150 of 725 posts
Earlier quoted context omitted.
Just yesterday, here on HN there was an article [1] about adversarial attacks that could make road signs get misread by ML recognition systems I'd be astonished if it wasn't possible to do the same thing here. [1] https://news.ycombinator.com/item?id=28204077
But the remarkable thing there (and with all other adversarial attacks I've seen) is that the ML classifier is fooled, while for us humans it is obvious that it is still the original image (if maybe slightly perturbed). But in the case of Apple's CSAM detection, the collision would first have to fool the victim into seeing an innocent picture and storing it (presumably, they would not accept and store actual CSAM [^]…
step 1 - As others have pointed out, there are plenty of ways of getting an image onto someone's phone without their explicit permission. WhatsApp (and I believe Messenger) do this by default; if someone sends you an image, it goes onto your phone and gets uploaded to iCloud.
step 2 - TFA proves that hash collision works, and fooling perceptual algorithms is already a known thing. This whole automatic screening process is known to be vulnerable already.
step 3 - Humans are harder to fool, but tech giants are not great at scaling human intervention; their tendency is to only use humans for exceptions because humans are expensive and unreliable. This is going to be a lowest-cost-bidder developing-country thing where the screeners are targeted on screening X images per hour, for a value of X that allows very little diligence. And the consequences of a false positive are probably going to be minimal - the screeners will be monitored for individual positive/negative rates, but that's about it. We've seen how this plays out for YouTube copyright claims, Google account cancellations, App store delistings, etc.
People's lives are going to be ruined because of this tech. I understand that children's lives are already being ruined because of abuse, but I don't see that this tech is going to reduce that problem. If anything it will increase it (because new pictures of child abuse won't be on the hash database).
Expectation : Political rivals and enemies of powerful people will be taken out because c-ild pornography will be found in their phone. Pegasus can already monitor and exfiltrate every ounce of data right now, it won't be that hard to insert compromising images on the infected device. Any news about "c-ild porn" being found on someone's phone is suspect now. This has been done before : 1) https://www.deccanchronicle.…
First CP. Then leaked or unauthorized nudes will get filtered. Filters for terrorists, and terrorist imagery or symbols. Start scanning for guns and drugs. Drug dealers and criminals get added to the list. How long before before it’s dissidents, political opponents, and minorities in dictatorships? How long before Tim Cooks CP filters are used against LGBT groups abroad?
Siri: "You seem to be having unpatriotic thoughts. We are dispatching someone to help you."
Earlier quoted context omitted.
All criminal accusations, including true ones , should be treated as false until the accused is proven guilty. This is a fundamental tenet of human rights in western, small-l liberal free societies. The fact that this is controversial these days is literally insane to me. The consequences of throwing this fundamental system out the window is that you get the sort of nonsense that happened with Assange, where he was l…
> All criminal accusations, including true ones, should be treated as false until the accused is proven guilty. No, they need to be treated as unproven, a very critical difference. Just to be clear, witness testimony, including testimony FROM THE VICTIM, is evidence of the crime. Just for some reason, in rape cases, we go all wonky with this principle.
Right. I have a jar of gumballs and tell you I think there is an even number of gumballs in it. Do you believe me? If you don't, does that mean you believe there are in fact an odd number of gumballs in it? No, you do not believe either that there are an odd or an even number, because you just don't know. For a criminal accusation , your initial belief should not be guilty or innocent, it should be, "I just don't know".
Earlier quoted context omitted.
All criminal accusations, including true ones , should be treated as false until the accused is proven guilty. This is a fundamental tenet of human rights in western, small-l liberal free societies. The fact that this is controversial these days is literally insane to me. The consequences of throwing this fundamental system out the window is that you get the sort of nonsense that happened with Assange, where he was l…
> All criminal accusations, including true ones, should be treated as false until the accused is proven guilty. No, they need to be treated as unproven, a very critical difference. Just to be clear, witness testimony, including testimony FROM THE VICTIM, is evidence of the crime. Just for some reason, in rape cases, we go all wonky with this principle.
In a criminal trial:
Victim: This person did it
Defendant: No I didn't
Not guilty
Apple's scheme includes operators manually verifying a low-res version of each image matching CSAM databases before any intervention. Of course, grey noise will never pass for CSAM and will fail that step. The fact that you can randomly manipulate random noise until it matches the hash of an arbitrary image is not surprising. The real challenge is generating a real image that could be mistaken for CSAM at low res + i…
Strongly disagree. (1) The primary feature of any decent hash function is that this should not happen. (2) Any preimage attack opens the way for further manipulations like you describe.
Earlier quoted context omitted.
All criminal accusations, including true ones , should be treated as false until the accused is proven guilty. This is a fundamental tenet of human rights in western, small-l liberal free societies. The fact that this is controversial these days is literally insane to me. The consequences of throwing this fundamental system out the window is that you get the sort of nonsense that happened with Assange, where he was l…
> All criminal accusations, including true ones, should be treated as false until the accused is proven guilty. No, they need to be treated as unproven, a very critical difference. Just to be clear, witness testimony, including testimony FROM THE VICTIM, is evidence of the crime. Just for some reason, in rape cases, we go all wonky with this principle.
People lie, people have memory issues. Complicating things is the lovely issue modernity has brought, of two adults getting intoxicated and fornicating followed by regret and rape accusations sometime later. Then we have weaponized accusations - just like we have keyboard warriors making false police reports to have their opposition receive a SWAT team visit, we have people that will make false accusations to get revenge after a perceived slight.
What is wonky to me is the people who hear an accusation and treat it like the Gospel, destroying the accused depriving them of any possible justice. This is what is truly, absolutely, criminally insane.
Earlier quoted context omitted.
All criminal accusations, including true ones , should be treated as false until the accused is proven guilty. This is a fundamental tenet of human rights in western, small-l liberal free societies. The fact that this is controversial these days is literally insane to me. The consequences of throwing this fundamental system out the window is that you get the sort of nonsense that happened with Assange, where he was l…
> All criminal accusations, including true ones, should be treated as false until the accused is proven guilty. No, they need to be treated as unproven, a very critical difference. Just to be clear, witness testimony, including testimony FROM THE VICTIM, is evidence of the crime. Just for some reason, in rape cases, we go all wonky with this principle.
2. The state is the only authorized monopoly of violence and they should treat unproven and untrue as identical, and the only place where that decision is made is in a courtroom.
3. The 'believe the victims' activists however are rightfully (IMHO) suggesting to break principle #2 because there is institutional and systemic supression of the rule of law being applied properly. I would consider this civil disobedience.
4. The state needs to get it's act together and administer the law properly. Only through reform can they regain the legitimacy that is required for #2.
5. Those reforms should focus on racism, sexism and classism. It should focus on what part of the law is too ambiguous for either law enforcement and the judiciary branch.
6. This might include actually blinding the court, i.e. offer only verifiable facts that are admissable and can not be utilized as widgets for race, gender or social economic positions.
If you don't think the legal system has a problem ask yourself why every lawyer recommends the defendant to wear a suit to court. How could it matter if the process was assumed to be without bias.
Earlier quoted context omitted.
Cross-posting from another thread [1]: 1. Obtain known CSAM that is likely in the database and generate its NeuralHash. 2. Use an image-scaling attack [2] together with adversarial collisions to generate a perturbed image such that its NeuralHash is in the database and its image derivative looks like CSAM. A difference compared to server-side CSAM detection could be that they verify the entire image, and not just the…
Right. So, sending actual CSAM would also work as an attack, but would be detected by the victim and could be corrected (delete images). But a conceivable novel avenue of attack would be to find an image that: 1. Does not look like CSAM to the innocent victim in the original 2. Does match known CSAM by NeuralHash 3. Does look like CSAM in the "visual derivative" reviewed by Apple, as you highlight.
I doubt deleting them (assuming the victim sees them) works once the image has been scanned. And, given that this probably comes with a sufficient smear campaign, deleting them will be portraye. as evidence of guilt