Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Under new law, cops bust famous cartoonist for AI-generated CSAM

arstechnica.com

Post reply on HN
Print view View original

51–60 of 87 posts

Re: Under new law, cops bust famous cartoonist for AI-generated CSAM

#51

Post by harshreality »

9rx wrote: post #44
defrost wrote: post #9

Earlier quoted context omitted.

> I think you would be hard-pressed to prove that any model trained on internet scrapes definitively wasn't trained on any CSAM whatsoever. I'd be hard pressed to prove that you definitely hadn't killed anybody ever. Legally if it's asserted that these images are criminal because they are the result of being the product of an LLM trained on sources that contained CSAM then the requirement would be to prove that asser…

> Legally if it's asserted that these images are criminal because they are the result of being the product of an LLM trained on sources that contained CSAM then the requirement would be to prove that assertion. Legally, possession of CSAM is against the law because there is an assumption that possession proves contribution to market demand, with an understanding that demand incentives production of supply, meaning th…

I'm somewhat sympathetic to that argument. However, it doesn't stop there.

Violent video games prove contribution to market demand for FPS-style videos of mass shootings or carjackings, so can/should we ban Call of Duty and Grand Theft Auto now?

(Note that the "market demand" argument is subtly different from the argument that the games directly cause people to become more violent, either in general or by encouraging specific copycat violence. Studies on [lack of] direct violence causation are weak and disputed.)

Re: Under new law, cops bust famous cartoonist for AI-generated CSAM

#52

Post by gs17 »

ashleyn wrote: post #34
danaris wrote: post #30

If it's AI-generated, it is fundamentally not CSAM. The reason we shifted to the terminology "CSAM", away from "child pornography", is specifically to indicate that it is Child Sexual Abuse Material: that is, an actual child was sexually abused to make it. You can call it child porn if you really want, but do not call something that never involved the abuse of a real, living, flesh-and-blood child "CSAM". (Or "CSEM"—…

This is where my technical knowledge of genAI breaks down, but wouldn't an image generator be unable to produce such imagery unless honest-to-god CSAM were used in the training of it?

It's like the early demo for DALL-E where you could get "an armchair in the shape of an avocado", which presumably wasn't in the training set, but enough was in it to generalize the "armchair" and "avocado" concepts and combine them.

Re: Under new law, cops bust famous cartoonist for AI-generated CSAM

#53

Post by harshreality »

metalcrow wrote: post #5
falcor84 wrote: post #2

> "The creation of CSAM using AI is inherently harmful to children because the machine-learning models utilized by AI have been trained on datasets containing thousands of depictions of known CSAM victims," it says, "revictimizing these real children by using their likeness to generate AI CSAM images into perpetuity." The word "inherently" there seems like a big stretch to me. I see how it could be harmful to them, b…

https://en.wikipedia.org/wiki/Relationship_between_child_por... is a good starting link on this. When i last checked, there were maybe 5 studies total (imagine how hard it is to get those approved by the ethics committees), all of which found different results, some totally the opposite of each other. Then again, it already seems clear that violent video games do not cause violence, and access to pornography does not…

The few studies on "video games cause violence" I've seen have been extremely limited in scope. They're too concerned with short-term frequency of playing particular games, or desire to play them. They're not concerned enough with the influence of being quite familiar with such games, or how cultural prevalence of such games normalizes thoughts of certain behaviors and shifts the Overton window. There are also selection bias problems. I'd expect media and games to more greatly affect people already psychologically unstable or on the criminal fringe... not people likely to be study participants.

Studies on sexual violence and changes in that over time have even more problems, for example how difficult it is to get average people to report accurately about their private relationships. Those people likely to volunteer accurate information are not necessarily representative.

Re: Under new law, cops bust famous cartoonist for AI-generated CSAM

#54

Post by yellowapple »

defrost wrote: post #7
falcor84 wrote: post #2

> "The creation of CSAM using AI is inherently harmful to children because the machine-learning models utilized by AI have been trained on datasets containing thousands of depictions of known CSAM victims," it says, "revictimizing these real children by using their likeness to generate AI CSAM images into perpetuity." The word "inherently" there seems like a big stretch to me. I see how it could be harmful to them, b…

There's a legally challengable assertion there; "trained on CSAM images". I imagine an AI image generation model could be readily trained on images of adult soldiers at war and images of children from instagram and then be used to generate imagery of children at war. I have zero interest in defending exploitation of children, the assertion that children had to have been exploited in order to create images of children…

Exactly. The abundance of AI-generated renditions of Shrimp Jesus doesn't mean it was trained on actual photos of an actual Shrimp Jesus.

Re: Under new law, cops bust famous cartoonist for AI-generated CSAM

#55

Post by ALittleLight »

WarOnPrivacy wrote: post #47
ALittleLight wrote: post #42

Earlier quoted context omitted.

You could say the same thing about real CSAM. Authorities claim you have it, present a hard drive they "recovered" from your property to court and you get convicted. At some level we just have to hope, and be skeptical of etc, that the government is basically staffed by mostly trustworthy people.

>> Of course, no real children are being abused, > You could say the same thing about real CSAM. Maybe this is trying to make a technical point. But abuse is a brutally heavy topic, mere technicalities aren't ever going to be strong enough to support the weight put upon them. That said: I do believe there are counter arguments to be made. I question the ethics and actions of people who leverage the pain of injured ch…

In the scenario I laid out, no additional real children are being abused either - because the government is simply framing you and you never consumed or produced any CSAM. That's what I meant by it being similar - if your concern with punishing people for AI generated CSAM is that law enforcement could lie about it and frame innocent people, then what I'm trying to point out is, that same risk exists with the real thing, they could lie about that too - if you want the government to enforce any laws you need to trust they're mostly trustworthy.

Re: Under new law, cops bust famous cartoonist for AI-generated CSAM

#56

Post by yellowapple »

ashleyn wrote: post #32
falcor84 wrote: post #2

> "The creation of CSAM using AI is inherently harmful to children because the machine-learning models utilized by AI have been trained on datasets containing thousands of depictions of known CSAM victims," it says, "revictimizing these real children by using their likeness to generate AI CSAM images into perpetuity." The word "inherently" there seems like a big stretch to me. I see how it could be harmful to them, b…

I always found these arguments to be contrived especially when it's already well-known that in the tradition of every Western government, there is no actual imperative for every crime to be linked directly to a victim. It's a far better argument to me, to suggest that the societal utility in using the material to identify and remove paedophiles before they have an actual victim far exceeds the utility of any sort of…

> It's a far better argument to me, to suggest that the societal utility in using the material to identify and remove paedophiles before they have an actual victim far exceeds the utility of any sort of "freedom" to such material.

Or at the very least: flag pedophiles for further investigation, including one's social network. Even if a given pedophile hasn't actually harmed any children, one probably is acquainted with other pedophiles, possibly ones who have harmed children.

Softening the punishment for CSAM possession in and of itself could actually help with investigating the creation of (non-simulated) CSAM by this logic, since people tend to be more cooperative to investigators when they think they have done nothing wrong and have nothing to hide.

Re: Under new law, cops bust famous cartoonist for AI-generated CSAM

#57

Post by yellowapple »

danaris wrote: post #30

If it's AI-generated, it is fundamentally not CSAM. The reason we shifted to the terminology "CSAM", away from "child pornography", is specifically to indicate that it is Child Sexual Abuse Material: that is, an actual child was sexually abused to make it. You can call it child porn if you really want, but do not call something that never involved the abuse of a real, living, flesh-and-blood child "CSAM". (Or "CSEM"—…

I think the one case where I'd disagree is when it's a depiction of an actual person - say, someone creates pornography (be it AI-generated, drawn, CG-animated, etc.) depicting a person who actually exists in the real world, and not just some invented character. That's certainly a case where it'd cross into actual CSAM/CSEM, because despite the child not physically being abused/exploited in the way depicted in the work, such a defamatory use of the child's likeness would constitute psychological abuse/exploitation.

Re: Under new law, cops bust famous cartoonist for AI-generated CSAM

#59

Post by 6ix8igth »

loeg wrote: post #50
NotYourLawyer wrote: post #33

Seems fine.

Right? Hard to be mad about the outcome.

Laws like this impede everyone while providing little tangible benefit for actual children. It's silly to imply that generating an image with a model hurts anyone.

In general I don't like legislation that tries to criminalize abstract depictions of something that would otherwise be a crime. It's too intangible, and opens the opportunity for abusing the legal system to persecute undesirables. We have to be really careful about what spiders we swallow to catch small (or sometimes imaginary) flies.

Re: Under new law, cops bust famous cartoonist for AI-generated CSAM

#60

Post by 9rx »

harshreality wrote: post #51
9rx wrote: post #44

Earlier quoted context omitted.

> Legally if it's asserted that these images are criminal because they are the result of being the product of an LLM trained on sources that contained CSAM then the requirement would be to prove that assertion. Legally, possession of CSAM is against the law because there is an assumption that possession proves contribution to market demand, with an understanding that demand incentives production of supply, meaning th…

I'm somewhat sympathetic to that argument. However, it doesn't stop there. Violent video games prove contribution to market demand for FPS-style videos of mass shootings or carjackings, so can/should we ban Call of Duty and Grand Theft Auto now? (Note that the "market demand" argument is subtly different from the argument that the games directly cause people to become more violent, either in general or by encouraging…

Tell us more about the market that is carrying out mass shootings/carjackings with intent to sell the imagery of it. This is the first I've heard of it. In fact, when mass shootings do occur they are usually explained away for some other reason (e.g. a troubled teen who went off the rails), not that it is the day-to-day operation of a videography business.
Post reply on HN