Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Teen boys use AI to make fake nudes of classmates, sparking police probe

arstechnica.com

Post reply on HN
Print view View original

61–70 of 90 posts

Re: Teen boys use AI to make fake nudes of classmates, sparking police probe

#61

Post by autoexec »

jltsiren wrote: post #51
autoexec wrote: post #6

If they'd cut out their classmate's head from their yearbook photo and pasted it onto the body of a pornstar with elmer's glue would police have gotten involved? This type of bullying is as old as dirt. Bullies suck, and the kids who did this should get in some kind of trouble with the school and their parents, but this is not something police need to be involved in. The moral panic over deepfakes is ridiculous, but…

The act of creating something is usually just irrelevant tinkering. Discussions like this on HN tend to overemphasize the importance of creation, probably because most of us have jobs where we are supposed to create something. But the real-world consequences of creation usually don't come from what was created or how it was created, but from distributing the creation. And that's what laws should be concerned about. C…

> Distributing it should be judged according to the potential harm

I'll admit that showing your buddies a deepfake/photoshop/cut and paste job of that one kid you hate from social studies is very different from creating a website or fake social media profile where the whole world can see it. In this case, it seems like some pics were shared with classmates who deleted the photos when they were caught and now the police can't even find copies to see what all the fuss was about, but the internet adds an opportunity for harms that extend far beyond the classroom.

There might be some instances where a situation requires intervention beyond parents and teachers, but I see no evidence that this was one of them.

Re: Teen boys use AI to make fake nudes of classmates, sparking police probe

#62

Post by hiisukun »

I have some experience with this in a previous job -- one of the commenters here is right: going back to before smart phones, there has always been cut & paste nudes/porn made by kids. From magazines, through jpgs, now even mkvs.

However, even back in the day there has been quite the spectrum of just how bad this kind of thing is: one image of a crush, who is 17 (technically underage), shared with a best friend? Inappropriate but not across the line of "law enforcement action" even if the crush's parent is very offended.

A google drive link with 100s of pornographic deepfakes, of a single person (or sometimes many students) from a school, many of them 12-16 years old, shared extensively across the internet? This is a proper serious problem. Kinda like the difference between throwing a rock at a bus vs detonating a bomb in a train station. Both are some amount of kinetic force, but the magnitude is wildly different.

I'm not pretending that physical violence is the same as making pornographic imagery, but my experience in that field (no jokes plz!) shows a much stronger link than many would like to admit, from one to the next. It's useful for teenagers to learn where these boundaries are in their particular society - life is often more pleasant when ~everyone agrees on the local standards of behaviour and they are (mostly) followed.

As for police action where I lived, it too followed a broad spectrum: on one end, there is arrest and court action which on occasion is definitely warranted (another pun, sorry). On the flipside, there's educating kids about the reality of the world they live in: If you share a nude, it might get shared again -- consider not including your face and your reproductive organs in the same picture. If someone generates a fake picture of you, it'll feel hurtful and you will be wronged, but what if they wrote something mean (and fake) about you? Eventually you will need to deal with the fact that some people are arses.

Some combination of protecting victims (but also helping them grow a thick skin in this world), educating kids on what is appropriate so they can know there is some boundary where things move from 'funny' to 'wrong'. They usually already have a sense of 'private' vs 'public', though that distinction is a becoming lot blurrier/different for current young generations.

And nothing above has anything really to do with 'AI'.

Re: Teen boys use AI to make fake nudes of classmates, sparking police probe

#63

Post by JumpCrisscross »

dragonwriter wrote: post #59
JumpCrisscross wrote: post #31

Earlier quoted context omitted.

> glued on photo will always be uncanny, but deepfake will not Eh, this is still a civil matter. Unless the picture was made for the purpose of harassment, there is no ill will we can presuppose the perpetrators had towards the victim. Just immense immaturity and a lack of judgement and empathy. Judgement and empathy can be taught. (If it was done with the intent of causing distress, i.e. cruelly, it's a matter for t…

> Eh, this is still a civil matter. Well, yes, that's clearly true whether or not it is also a criminal matter. > Unless the picture was made for the purpose of harassment, there is no ill will we can presuppose the perpetrators had towards the victim That is almost tautologically true, but on what bases do you conclude that it is so certain as not to warrant inquiry that there was no intent to harrass?

> on what bases do you conclude that it is so certain as not to warrant inquiry that there was no intent to harrass

I conclude it is uncertain enough to warrant solely civil engagement on the basis of them being kids. Otherwise, every five year old pantsing their sibling becomes a matter for armed law enforcement.

Re: Teen boys use AI to make fake nudes of classmates, sparking police probe

#64

Post by underseacables »

autoexec wrote: post #6

If they'd cut out their classmate's head from their yearbook photo and pasted it onto the body of a pornstar with elmer's glue would police have gotten involved? This type of bullying is as old as dirt. Bullies suck, and the kids who did this should get in some kind of trouble with the school and their parents, but this is not something police need to be involved in. The moral panic over deepfakes is ridiculous, but…

I seem to recall, there was a case where a man was convicted of child pornography, because he cut out pictures of children's faces, and glued them onto the bodies of adult porn actors.

Re: Teen boys use AI to make fake nudes of classmates, sparking police probe

#65

Post by autoexec »

underseacables wrote: post #64
autoexec wrote: post #6

If they'd cut out their classmate's head from their yearbook photo and pasted it onto the body of a pornstar with elmer's glue would police have gotten involved? This type of bullying is as old as dirt. Bullies suck, and the kids who did this should get in some kind of trouble with the school and their parents, but this is not something police need to be involved in. The moral panic over deepfakes is ridiculous, but…

I seem to recall, there was a case where a man was convicted of child pornography, because he cut out pictures of children's faces, and glued them onto the bodies of adult porn actors.

That seems kind of wrong to me too, but the main thing that differentiates it is that the person in that case was an adult. We know that kids are going to do stupid things and childhood is the time to teach them to be better. It's not helping anyone to ruin a kid's life for acting like a child.

Re: Teen boys use AI to make fake nudes of classmates, sparking police probe

#66

Post by dragonwriter »

JumpCrisscross wrote: post #63
dragonwriter wrote: post #59

Earlier quoted context omitted.

> Eh, this is still a civil matter. Well, yes, that's clearly true whether or not it is also a criminal matter. > Unless the picture was made for the purpose of harassment, there is no ill will we can presuppose the perpetrators had towards the victim That is almost tautologically true, but on what bases do you conclude that it is so certain as not to warrant inquiry that there was no intent to harrass?

> on what bases do you conclude that it is so certain as not to warrant inquiry that there was no intent to harrass I conclude it is uncertain enough to warrant solely civil engagement on the basis of them being kids. Otherwise, every five year old pantsing their sibling becomes a matter for armed law enforcement.

Yeah, no, that doesn't follow. "Kids" aren't a uniform category; there's a reason that kids under 7 are generally viewed as incapable of the necessary mental capacity for criminal accountability, and between 7 and 14 there is in most states a presumption of incapacity that must be overcome in addition to the usual proof requirements for crime. So, no, you don't have to treat acts by minors in their late teens as irrefutably noncriminal without investigation to avoid considering acts by five year olds to be that way (even before considering ways that the act you suggest for the five year old was dissimilar to the one at issue for the older minors in the article, which would be another axis on which treatment could also reasonably be differentiated.)

Re: Teen boys use AI to make fake nudes of classmates, sparking police probe

#67

Post by mcphage »

nvy wrote: post #53
JumpCrisscross wrote: post #49

Earlier quoted context omitted.

> society is going to have to come around to the fact that this technology can't be legislated away Neither can fraud or murder. Yet we still ban it. This concept that if a problem cannot be 100% solved by the law, it shouldn't be banned, is an umbrella for the insufferable.

By all means ban it. I'm not suggesting that 100% is required. I'm suggesting that banning it won't really accomplish much because it's pretty hard to enforce what you can and can't generate on your own machine without trampling on people's civil rights.

I don't think it matters what you generate on your own machine, but it does matter what you share with other people.

Re: Teen boys use AI to make fake nudes of classmates, sparking police probe

#69

Post by mikotodomo »

autoexec wrote: post #6

If they'd cut out their classmate's head from their yearbook photo and pasted it onto the body of a pornstar with elmer's glue would police have gotten involved? This type of bullying is as old as dirt. Bullies suck, and the kids who did this should get in some kind of trouble with the school and their parents, but this is not something police need to be involved in. The moral panic over deepfakes is ridiculous, but…

Whaaaaaaa? How should it be legal to imagine minors naked? And in this case there is proof in the form of the child pornography they distributed.

Re: Teen boys use AI to make fake nudes of classmates, sparking police probe

#70

Post by averageRoyalty »

mikotodomo wrote: post #69
autoexec wrote: post #6

If they'd cut out their classmate's head from their yearbook photo and pasted it onto the body of a pornstar with elmer's glue would police have gotten involved? This type of bullying is as old as dirt. Bullies suck, and the kids who did this should get in some kind of trouble with the school and their parents, but this is not something police need to be involved in. The moral panic over deepfakes is ridiculous, but…

Whaaaaaaa? How should it be legal to imagine minors naked? And in this case there is proof in the form of the child pornography they distributed.

> How should it be legal to imagine minors naked?

Dicey grounds here, but it should be legal to imagine _anything_. Thoughtcrimes are always wrong, and there never should be any law on what you can think or imagine.

Post reply on HN