Other than being randomly happening, it is not a threat. ElevenLabs does it in a few seconds already. If you are using mic while using Ai, expect companies like OpenAi to steal every bit of your soul. Nothing new here.
ChatGPT unexpectedly began speaking in a user's cloned voice during testing
111–120 of 164 posts
Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing
#112Earlier quoted context omitted.
You are making a very popular mistake: confusing the training procedure for LLMs (autocomplete) with "how they work"/their internal ontology (mostly unknown). When we teach children how to do arithmetic, we have them predict missing items in equations. We don't accuse them of "only doing autocomplete". The same applies for LLMs.
This is only a common mistake to the subset of people who labor under the delusion that we have actually stumbled upon Generalized Artificial Intelligence. We know how LLMs work fundamentally and what their limits are. LLMs are only able to make “correct sounding” statements which have the side effect of being correct a certain percentage of the time. They do not have the ability to reason nor engage in high level th…
> We know how LLMs work fundamentally
We know how they work only at the lowest level (the arithmetic operations) and the highest level (the optimization criterion and the representation of various layers, like the input/output layer and for things we can easily probe like embedding matrices).
We do not know "what they are doing" on the inner layers. This is an area of active research.
> They do not have the ability to reason nor engage in high level thought.
You are speculating (and probably incorrectly), or you're really holding back some valuable research from the field of AI interpretability.
Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing
#113Early on in the voice public release, I asked my 15 year old to give it a shot. They (they’re NB) had a long meandering conversation about their favorite book series, the Percy Jackson series.
At about 15 mins into the conversation between my kiddo and ChatGPT, the model started to take on the vocal mannerisms of my kiddo. It started using more “umms” and “you knows.”
At first this felt creepy but as I explained it to my kid, it’s because their own text has become weighted enough in the token count for the LLM to start incorporating or/and somewhere in the embedded prompts is “empathize with the user and emphasize clarity” and that prompting meant mirroring back speech styles.
This is exactly the same as that only with audio.
Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing
#114Earlier quoted context omitted.
A couple seconds? Which project does this?
every single voice cloning project I'm sure you could have used google https://elevenlabs.io/app/voice-lab https://app.resemble.ai/users/sign_in https://github.com/neonbjb/tortoise-tts https://coqui.ai/blog/tts/open_xtts it was even possible 5 years ago https://github.com/CorentinJ/Real-Time-Voice-Cloning
Nope. If you actually tried those, you would quickly find out they don't work. It's actually really hard to clone a voice from a few seconds sample.
Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing
#115Earlier quoted context omitted.
> I'm literally describing how inference is done. You are very clearly not doing that. Nothing about your comment had anything to do with the internal structure of LLMs. I believe you that you've set up some models with pytorch or whatever, but this seemingly hasn't translated to a sufficiently coherent mental model to make the distinction between the extrinsic optimization criterion and intrinsic behavior.
I'm not talking about optimization criteria or training, I'm talking about how we use the model for inference. We feed in a context and it gives a probability distribution for the next word. We sample from that distribution following some set of rules. We then update the context with the new word and repeat. That algorithm is an autocomplete algorithm. As long as that's what LLM inference looks like, all problems tha…
> all problems that we want to feed to an LLM therefore must be translated to autocomplete.
I don't disagree with this, but I do disagree with this earlier statement:
> The further you get from autocomplete, the less reliable the resulting product
Any naturally sequential problem is trivial to translate to autocomplete with minimal loss of fidelity.
Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing
#116Earlier quoted context omitted.
A couple seconds? Which project does this?
every single voice cloning project I'm sure you could have used google https://elevenlabs.io/app/voice-lab https://app.resemble.ai/users/sign_in https://github.com/neonbjb/tortoise-tts https://coqui.ai/blog/tts/open_xtts it was even possible 5 years ago https://github.com/CorentinJ/Real-Time-Voice-Cloning
how many do you think "a couple" is?
Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing
#117Earlier quoted context omitted.
The big difference is, there was a very high bar to forging photographs, and most news was gated (with the ability to easily find, and sue those guilty of slander/libel). Now it's utterly simplistic to forge, to libel, to slander, and there is no easy path in many cases to sue. While you can say "yes, but..." to the above, that's the reality that we've lived with for 150 years, less extremely rare edge cases. All thi…
> The same is happening with all content on the internet. Realistically the web is now dead. It's now gone. Even things such as wikipedia are going to die, as over the next 2 to 3 years LLM output will become utterly and completely indistinguishable in all aspects. Like I said, people have been saying this exact thing for a couple years now. I'm sure I'll be hearing the same in a couple more.
Compared to the 90s, and the 00s, the web is useless.
Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing
#118Earlier quoted context omitted.
I'm not talking about optimization criteria or training, I'm talking about how we use the model for inference. We feed in a context and it gives a probability distribution for the next word. We sample from that distribution following some set of rules. We then update the context with the new word and repeat. That algorithm is an autocomplete algorithm. As long as that's what LLM inference looks like, all problems tha…
> all problems that we want to feed to an LLM therefore must be translated to autocomplete. I don't disagree with this, but I do disagree with this earlier statement: > The further you get from autocomplete, the less reliable the resulting product Any naturally sequential problem is trivial to translate to autocomplete with minimal loss of fidelity.
In other words, would it be fair to say that any naturally sequential problem is not very far from autocomplete?
Again, I think you're putting words in my mouth and thoughts in my head that aren't there. A lot of people have reacted to AI hype by going the other way and underestimating them—that's not me. I think there are lots of problems they can solve, I just think they all boil down to autocomplete and if you can't boil it down to autocomplete you're not ready to implement it yet with an LLM.
Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing
#119Earlier quoted context omitted.
We have adapted to monumental shifts in how we develop trust in society for as long as society has existed - from the printing press to photography to the Internet to CGI to ... I don't see this as any different. We will determine new ways of establishing trust. They'll certainly have flaws, as establishing trust in a society always has, but we'll learn to recognize those flaws and hopefully fix them. Beyond that, wh…
The big difference is, there was a very high bar to forging photographs, and most news was gated (with the ability to easily find, and sue those guilty of slander/libel). Now it's utterly simplistic to forge, to libel, to slander, and there is no easy path in many cases to sue. While you can say "yes, but..." to the above, that's the reality that we've lived with for 150 years, less extremely rare edge cases. All thi…
IMO will just shift power back to journalists. Not the worst outcome.
Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing
#120Earlier quoted context omitted.
This is only a common mistake to the subset of people who labor under the delusion that we have actually stumbled upon Generalized Artificial Intelligence. We know how LLMs work fundamentally and what their limits are. LLMs are only able to make “correct sounding” statements which have the side effect of being correct a certain percentage of the time. They do not have the ability to reason nor engage in high level th…
> We know how LLMs work fundamentally We know how they work only at the lowest level (the arithmetic operations) and the highest level (the optimization criterion and the representation of various layers, like the input/output layer and for things we can easily probe like embedding matrices). We do not know "what they are doing" on the inner layers. This is an area of active research. > They do not have the ability t…
You talk about about as if a human-created neural network is at the same level as quantum physics where there are limits as to our understanding. We know very well how large language models work even if the capabilities of this technology are actively being explored.
You along with others here are far overstating the unknowns we have within the context of AI, whether this is the result of a misinformation campaign targeted at trying to boost the value of this tech or if the pop-sci takes have really gotten too prevalent, it is unclear to me.