Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Under new law, cops bust famous cartoonist for AI-generated CSAM

arstechnica.com

Post reply on HN
Print view View original

21–30 of 87 posts

Re: Under new law, cops bust famous cartoonist for AI-generated CSAM

#21

Post by Hizonner »

falcor84 wrote: post #2

> "The creation of CSAM using AI is inherently harmful to children because the machine-learning models utilized by AI have been trained on datasets containing thousands of depictions of known CSAM victims," it says, "revictimizing these real children by using their likeness to generate AI CSAM images into perpetuity." The word "inherently" there seems like a big stretch to me. I see how it could be harmful to them, b…

The word "revictimizing" seems like an even bigger stretch. Assuming the output images don't actually look like them personally (and they won't), how exactly are they more victimized than anybody else in the training data? Those other people's likenesses are also "being used to generate AI CSAM images into perpetuity"... in a sort of attenuated way that's hard to even find if you're not desperately trying to come up with something.

The cold fact is that people want to outlaw this stuff because they find it icky. Since they know it's not socially acceptable (quite yet) to say that, they tend to cast about wildly until they find something to say that sort of sounds like somebody is actually harmed. They don't think critically about it once they land on a "justification". You're not supposed to think critically about it either.

Re: Under new law, cops bust famous cartoonist for AI-generated CSAM

#23

Post by Hizonner »

jsheard wrote: post #8
defrost wrote: post #7

Earlier quoted context omitted.

There's a legally challengable assertion there; "trained on CSAM images". I imagine an AI image generation model could be readily trained on images of adult soldiers at war and images of children from instagram and then be used to generate imagery of children at war. I have zero interest in defending exploitation of children, the assertion that children had to have been exploited in order to create images of children…

It is known that the LAION dataset underpinning foundation models like Stable Diffusion contained at least a few thousand instances of real-life CSAM at one point. I think you would be hard-pressed to prove that any model trained on internet scrapes definitively wasn't trained on any CSAM whatsoever. https://www.theverge.com/2023/12/20/24009418/generative-ai-i...

I think you'd be hard-pressed to prove that a few thousand images (out of over 5 billion in the case of that particular data set) had any meaningful effect on the final model capabilities.

Re: Under new law, cops bust famous cartoonist for AI-generated CSAM

#25

Post by metalcrow »

skissane wrote: post #18
userbinator wrote: post #13

Earlier quoted context omitted.

Wrong country (at least for now...!)

Not wrong country. In 2021, a Texas man was sentenced to 40 years in federal prison over CSAM text and drawings - https://www.justice.gov/opa/pr/texas-man-sentenced-40-years-...

Specifically, this ruling does not make that kind of content illegal. This person was convicted under federal obscenity statues, which are....fuzzy to say the least. As Supreme Court justice Potter Stewart said, it's a "i know it when i see it" thing.

Which in effect is basically a "you can go to jail if we think you're too gross" law.

Re: Under new law, cops bust famous cartoonist for AI-generated CSAM

#26

Post by jsheard »

defrost wrote: post #9
jsheard wrote: post #8

Earlier quoted context omitted.

It is known that the LAION dataset underpinning foundation models like Stable Diffusion contained at least a few thousand instances of real-life CSAM at one point. I think you would be hard-pressed to prove that any model trained on internet scrapes definitively wasn't trained on any CSAM whatsoever. https://www.theverge.com/2023/12/20/24009418/generative-ai-i...

> I think you would be hard-pressed to prove that any model trained on internet scrapes definitively wasn't trained on any CSAM whatsoever. I'd be hard pressed to prove that you definitely hadn't killed anybody ever. Legally if it's asserted that these images are criminal because they are the result of being the product of an LLM trained on sources that contained CSAM then the requirement would be to prove that asser…

Framing it in that way is essentially a get out of jail free card - anyone caught with CSAM can claim it was AI generated by a "clean" model, and how would the prosecution ever be able to prove that it wasn't?

I get where you are coming from but it doesn't seem actionable in any way that doesn't effectively legalize CSAM possession, so I think courts will have no choice but to put the burden of proof on the accused. If you play with fire then you'd better have the receipts.

Re: Under new law, cops bust famous cartoonist for AI-generated CSAM

#27

Post by Hizonner »

defrost wrote: post #7
falcor84 wrote: post #2

> "The creation of CSAM using AI is inherently harmful to children because the machine-learning models utilized by AI have been trained on datasets containing thousands of depictions of known CSAM victims," it says, "revictimizing these real children by using their likeness to generate AI CSAM images into perpetuity." The word "inherently" there seems like a big stretch to me. I see how it could be harmful to them, b…

There's a legally challengable assertion there; "trained on CSAM images". I imagine an AI image generation model could be readily trained on images of adult soldiers at war and images of children from instagram and then be used to generate imagery of children at war. I have zero interest in defending exploitation of children, the assertion that children had to have been exploited in order to create images of children…

> There's a legally challengable assertion there; "trained on CSAM images".

"Legally challengable" only in a pretty tenuous sense that's unlikely to ever haven any actual impact.

That'll be something that's recited as a legislative finding. It's not an element of the offense; nobody has to prove that "on this specific occasion the model was trained in this or that way".

It could theoretically have some impact on a challenge to the constitutionality of the law... but only under pretty unlikely circumstances. First you'd have to get past the presumption that the legislature can make any law it likes regardless of whether it's right about the facts (which, in the US, probably means you have to get courts to take the law under strict scrutiny, which they hate to do). Then you have to prove that that factual claim was actually a critical reason for passing the law, and not just a random aside. Then you have to prove that it's actually false, overcoming a presumption that the legislature properly studied the issue. Then maybe it matters.

I may have the exact structure of that a bit wrong, but that's the flavor of how these things play out.

Re: Under new law, cops bust famous cartoonist for AI-generated CSAM

#28

Post by alwa »

grapesodaaaaa wrote: post #19

Question for anyone that knows, but what happens if a 16 year old draws a naked picture of their 16 year old partner?

Last I heard, depending on local attitudes, potentially jail. For that matter, minors can be charged for producing images of themselves. The CSAM laws in the US tend not to draw distinctions between first- and third-party production:

https://www.ojp.gov/ncjrs/virtual-library/abstracts/blurring...

https://www.innocentlivesfoundation.org/threat-of-sg-csam/

https://www.findlaw.com/criminal/criminal-charges/child-porn...

Re: Under new law, cops bust famous cartoonist for AI-generated CSAM

#29

Post by Hizonner »

grapesodaaaaa wrote: post #19

Question for anyone that knows, but what happens if a 16 year old draws a naked picture of their 16 year old partner?

Depends on where you are. In some places in the world, into the gulag with them.

In fact, they can get gulaged if they draw a picture of a totally imaginary 16 year old, let alone a real person. It's kind of in dispute in the US, but that'd be the way to bet. 'Course, in a lot of US states, they're also in trouble for having a "partner" at all.

In practice, they'd probably have to distribute the picture somehow to draw the attention to get them arrested, and anyway they'd be a low priority compared to people producing, distributing, or even consuming, you know, real child porn... but if the right official decided to make an example of them, they'd be hosed.

Re: Under new law, cops bust famous cartoonist for AI-generated CSAM

#30

Post by danaris »

If it's AI-generated, it is fundamentally not CSAM.

The reason we shifted to the terminology "CSAM", away from "child pornography", is specifically to indicate that it is Child Sexual Abuse Material: that is, an actual child was sexually abused to make it.

You can call it child porn if you really want, but do not call something that never involved the abuse of a real, living, flesh-and-blood child "CSAM". (Or "CSEM"—"Exploitation" rather than "Abuse"—which is used in some circles.) This includes drawings, CG animations, written descriptions, videos where such acts are simulated with a consenting (or, tbh, non consenting—it can be horrific, illegal, and unquestionably sexual assault without being CSAM) adult, as well as anything AI-generated.

These kinds of distinctions in terminology are important, and yes I will die on this hill.

Post reply on HN