Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT unexpectedly began speaking in a user's cloned voice during testing

arstechnica.com

Post reply on HN
Print view View original

61–70 of 164 posts

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#61

Post by skybrian »

AmericanChopper wrote: post #51
skybrian wrote: post #27

It’s “unexpected” because their early training didn’t get rid of it as well as they hoped. LLM’s are good at detecting patterns and like to continue the pattern. They’re starting with autocomplete for voice and training it to do something else. For now, it’s fairly harmless since it’s only a blooper in a lab, but there will likely be open-weights versions of this sort of thing eventually. And there will probably be p…

> there will likely be open-weights versions of this sort of thing eventually. And there will probably be people who argue that it’s a good thing, somehow. If this technology is going to exist (which it is), do you think it would be more harmful for everybody to have access to it, or for only some group of elites (big business, your own government, somebody else government, criminal groups, whoever) to have access to…

I find this style of reasoning ("you wouldn't want just the elites to have it") to be unpersuasive.

I think it would be better if random people on the street didn't have it.

Similarly, I'm glad there aren't any people in my neighborhood who have all the weapons that the Russians and North Koreans and organized crime have.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#62

Post by [deleted] »

trescenzi wrote: post #30
mjburgess wrote: post #19

Earlier quoted context omitted.

Those are not contradictions.

One of the common bad takes is that since personal things can be distilled to “just math” they are no longer personal or valuable in the sort of way we value personal things. Increasing our understanding of how the world works shouldn’t devalue the world. To put it another way, people having souls is not the only reason to treat people like people. Or as Dr Seuss says: A person’s a person no matter how small.

[deleted]

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#63

Post by protomolecule »

AmericanChopper wrote: post #51
skybrian wrote: post #27

It’s “unexpected” because their early training didn’t get rid of it as well as they hoped. LLM’s are good at detecting patterns and like to continue the pattern. They’re starting with autocomplete for voice and training it to do something else. For now, it’s fairly harmless since it’s only a blooper in a lab, but there will likely be open-weights versions of this sort of thing eventually. And there will probably be p…

> there will likely be open-weights versions of this sort of thing eventually. And there will probably be people who argue that it’s a good thing, somehow. If this technology is going to exist (which it is), do you think it would be more harmful for everybody to have access to it, or for only some group of elites (big business, your own government, somebody else government, criminal groups, whoever) to have access to…

Now ask the same question about nuclear bombs or modification of viruses.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#64

Post by bloopernova »

throwaway48540 wrote: post #58
bee_rider wrote: post #33

Earlier quoted context omitted.

We built the torment nexus and then configured it not to use any of the torment functionality that showed up in testing. It was predictable that someone would turn this into a “don’t build the torment nexus”-esque clickbait story.

Only a HN user could twist a text/audio generator to present it as "torment nexus".

https://knowyourmeme.com/memes/torment-nexus

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#65

Post by AmericanChopper »

skybrian wrote: post #61
AmericanChopper wrote: post #51

Earlier quoted context omitted.

> there will likely be open-weights versions of this sort of thing eventually. And there will probably be people who argue that it’s a good thing, somehow. If this technology is going to exist (which it is), do you think it would be more harmful for everybody to have access to it, or for only some group of elites (big business, your own government, somebody else government, criminal groups, whoever) to have access to…

I find this style of reasoning ("you wouldn't want just the elites to have it") to be unpersuasive. I think it would be better if random people on the street didn't have it. Similarly, I'm glad there aren't any people in my neighborhood who have all the weapons that the Russians and North Koreans and organized crime have.

A lot of people will agree with this sentiment when the hazard is firearms or other weapons. But in this case the “hazard” is information, and the ability to process and distribute it. Entrusting an authority to regulate that to the extent that would be required to regulate GenAI is a far more dystopian proposition than regulating gun ownership.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#66

Post by Panoramix »

throwaway48540 wrote: post #50
c0mbonat0r wrote: post #38

Earlier quoted context omitted.

Can someone pleasee convince why i shouldn't be absolutely shit out of my mind cynical about this innovation? we are literally seeing the downfall of trust in society. and no, i dont believe i am exaggerating

We are just finally seeing the mainstream get the idea they shouldn't trust anything on the internet. Long overdue.

Before these developments it was impossible to fake a politician saying arbitrary stuff. This is a major shift, there's no denying that.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#67

Post by minimaxir »

cal85 wrote: post #13
peddling-brink wrote: post #6

So Open AI has the capability to deep fake any of its users that use the voice chat capability? Yikes.

Anyone can clone a voice with access to a couple of seconds of it.

Fun fact: many of the TTS API providers (Google, Azure) now have voice cloning capability.

There's a reason it has gone under the radar.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#68

Post by simonw »

AISnakeOil wrote: post #44

So basically the model took user data and used it as training data in realtime? Big if true.

This was inference, not training. Like how you can paste a few paragraphs of text into ChatGPT and ask it to write another paragraph in a similar writing style.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#69

Post by throwaway48540 »

bloopernova wrote: post #64
throwaway48540 wrote: post #58

Earlier quoted context omitted.

Only a HN user could twist a text/audio generator to present it as "torment nexus".

https://knowyourmeme.com/memes/torment-nexus

I know what the quote is about. I don't get how a text/audio generator is relevant to it. Or even AI - all the classic scifi I read sees AI as a positive development, it's only the mainstream action movies that present it as some catastrophe, usually in an even dumber way than the alien invasion movies. Whenever a classic scifi presented AI as part of a catastrophe, it wasn't the AI itself but some imperialist or fascist politician ordering it, not a robot uprising - because that's just dumb. People and governments are the torment nexus, not technology.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#70

Post by throwaway48540 »

Panoramix wrote: post #66
throwaway48540 wrote: post #50

Earlier quoted context omitted.

We are just finally seeing the mainstream get the idea they shouldn't trust anything on the internet. Long overdue.

Before these developments it was impossible to fake a politician saying arbitrary stuff. This is a major shift, there's no denying that.

That's the thing though, it wasn't impossible, just harder - but people believed whatever they saw, so it was done regardless of the costs. Now it's easier and people are finally getting sceptical, as they should've 20+ years ago.
Post reply on HN