Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

They stole my voice with AI

jeffgeerling.com

Post reply on HN
Print view View original

151–160 of 461 posts

Re: They stole my voice with AI

#151

Post by vnorilo »

microtonal wrote: post #141
godelski wrote: post #140

Earlier quoted context omitted.

> what if AI was used to imitate a person saying something blasphemeous? I've been contemplating writing an open letter to Dang to nuke my account. Because at this time you can likely deanonymize any user with a fair amount of comments. As long as you can correlate. You can certainly steal their language, even if not 100% accurate. It may be caution, but it isn't certain that we won't enter a dark forest and there's…

There should be a way to cryptographically sign posts (everywhere). I know, building a web of trust sucks, etc. But if there was someone with your username signing with a particular key for 10 years and then suddenly there is something controversial somewhere with a different key, something fishy is going on. Of course, this could be misused to post something with plausible deniability, but if you want to say somethi…

Not sure the lynch mob would pause to check that the web of trust is unbroken.

Re: They stole my voice with AI

#153

Post by kossTKR »

godelski wrote: post #140
ryzvonusef wrote: post #135

Everyone has their own fears about AI, but my fears are especially chilling; what if AI was used to imitate a person saying something blasphemeous? My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. And thi…

> what if AI was used to imitate a person saying something blasphemeous? I've been contemplating writing an open letter to Dang to nuke my account. Because at this time you can likely deanonymize any user with a fair amount of comments. As long as you can correlate. You can certainly steal their language, even if not 100% accurate. It may be caution, but it isn't certain that we won't enter a dark forest and there's…

Yep, im sure lots of people have written a lot of random stuff on a lot of forums that should absolutely stay anonymous from gossip to family secrets to honesty about career/workplace and what not.

If stylometric analysis runs on all comments on the internet then yeah.

Bad things will happen, very very bad.

I honestly think it should be at least illegal to do this kind of analysis because it'll be a treasure trove for the commercial sector to mine this data correlated to real people not to think of the destruction in millions of people with personal anonymous blogs etc.

Actually thinking about it further you could also easily group people political affiliations, and all kinds of other thoughts, dark, dark stuff!

Re: They stole my voice with AI

#154

Post by pjc50 »

vasco wrote: post #138
ryzvonusef wrote: post #135

Everyone has their own fears about AI, but my fears are especially chilling; what if AI was used to imitate a person saying something blasphemeous? My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. And thi…

The best we can hope for is that one personally avoids this for the first 5 years or so, and then it gets so widespread and easy that everyone will start doubting any videos they watch. The same way it took social media like reddit a few years of "finding the culprit" / "name and shame" till mods figured out that many times the online mob gets it wrong and so now that is usually not allowed. But many people will suff…

> everyone will start doubting any videos they watch.

This kills the medium.

Just as ubiquitous scam calls have moved people away from phones, this moves people away from using media which cannot be trusted. Done enough this destroys reporting and therefore democracy. I wonder when the first nonexistent candidate will be elected.

Re: They stole my voice with AI

#155

Post by microtonal »

godelski wrote: post #149
microtonal wrote: post #141

Earlier quoted context omitted.

There should be a way to cryptographically sign posts (everywhere). I know, building a web of trust sucks, etc. But if there was someone with your username signing with a particular key for 10 years and then suddenly there is something controversial somewhere with a different key, something fishy is going on. Of course, this could be misused to post something with plausible deniability, but if you want to say somethi…

> There should be a way to cryptographically sign posts (everywhere). This just confirms it is me. Which yes, reduces the problem of me being replicated, but does not do anything for my anonymity. That part may not seem important to you, considering you use your real name, but it is to me. It allows me to be more open. Key signing will not be the solution because it isn't addressing the problem. It exacerbates it.

This just confirms it is me. Which yes, reduces the problem of me being replicated, but does not do anything for my anonymity. That part may not seem important to you, considering you use your real name, but it is to me. It allows me to be more open.

Ah, think I see your point. Your worry is that language use, etc. could be use to deanonymize you, by correlating with text that was not written anonymously. But that's a separate issue from voice or writing style cloning to pretend it's you that said it. In the latter case you could use a pseudonymous signing key?

I agree that deanonymization is an issue that is hard to tackle. I wonder if someone studied how unique writing style is. E.g. browser fingerprints are fairly unique, but I wonder to what extend you can filter a person from, say a pool of 100 million, using writing style alone (grammar, vocabulary use, etc.). I guess it becomes quite easy if you engage in a lot of domain-specific discussions and use their vocabularies.

E.g. if I'd talk about Marlin-kernels here, you could probably narrow me down to a few hundred people. Throw in another comment about the Glove80. Maybe ten people at most?

Re: They stole my voice with AI

#156

Post by ryzvonusef »

godelski wrote: post #140
ryzvonusef wrote: post #135

Everyone has their own fears about AI, but my fears are especially chilling; what if AI was used to imitate a person saying something blasphemeous? My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. And thi…

> what if AI was used to imitate a person saying something blasphemeous? I've been contemplating writing an open letter to Dang to nuke my account. Because at this time you can likely deanonymize any user with a fair amount of comments. As long as you can correlate. You can certainly steal their language, even if not 100% accurate. It may be caution, but it isn't certain that we won't enter a dark forest and there's…

The problem I fear is that, let's say you once had a facebook account, we all deactivated our accounts when there was wave against Zuck a few years back, but as we know, facebook doesn't really delete your account.

Now imagine that account was linked to a SIM. It's trivial for a nefarious actor to get it re-activated, infact there was a video by Veritasium just today where they didn't even need your SIM.

But even if they are not that hi-tech, it's not that hard to get a SIM issued in your name, or other hacks of a similar nature, we have all heard of stories.

Worse, you lost that SIM a decade back, the number gets back into the queue, and is eventually re-issued to someone new... and they try to create a facebook account, and are presented with yours.

They can then re-activate your old facebook account, and post a video/audio/text of "godelski" saying they like pineapple on pizza. and before you can defend yourself, the pizzarias have lynched you.

(I dare not use a real example even as a jest, I live here)

Are you 100% sure of all your old social media accounts, all the SIM you have ever used to log-in to accounts?

We leave a long trail.

Re: They stole my voice with AI

#158

Post by aktenlage »

godelski wrote: post #140
ryzvonusef wrote: post #135

Everyone has their own fears about AI, but my fears are especially chilling; what if AI was used to imitate a person saying something blasphemeous? My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. And thi…

> what if AI was used to imitate a person saying something blasphemeous? I've been contemplating writing an open letter to Dang to nuke my account. Because at this time you can likely deanonymize any user with a fair amount of comments. As long as you can correlate. You can certainly steal their language, even if not 100% accurate. It may be caution, but it isn't certain that we won't enter a dark forest and there's…

Another solution would be to use an LLM to rephrase your posts, wouldn't it?

Not a great outlook though, if everybody does this...

Re: They stole my voice with AI

#159

Post by firtoz »

ryzvonusef wrote: post #135

Everyone has their own fears about AI, but my fears are especially chilling; what if AI was used to imitate a person saying something blasphemeous? My country is already has blasphemy lynching mobs based on the slightest perceived insult, real or imagined. They will mob you, lynch you, burn your corpse, then distribute sweets while you family hide and issue video messages denouncing you and forgiving the mob. And thi…

> You can say a lot of things about 'oh backward countries' but this will not stay there, this will spread

I'm sorry, but this is a cope out. The "lynching from apparent cultural deviation" is something that needs to be moved on from. Developed countries do the same too to some extent, with "cancel culture" and such.

There are ways to have progress in this, and, well, to feed someone's entrepreneurial spirit, it's one of those really hard problems that a lot of people, let's say, "a growing niche market", needs it to be solved.

Re: They stole my voice with AI

#160

Post by sandreas »

This is exactly the reason why I'm not open sourcing a tool I developed where you can take an audio book together with an epub to build an ljspeech dataset and train a voice model.

Although it was not too hard to create I believe making it easier is something i don't like to achieve...

I hate to say this but ruining a narrators existence with AI seems to get easier every day.

Post reply on HN