ChatGPT unexpectedly began speaking in a user's cloned voice during testing • Hacker News Board

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#161

Post by danielmarkbruce » Mon, Aug 12, 2024, 7:43 PM UTC

ionwake wrote: post #131

danielmarkbruce wrote: post #96

Earlier quoted context omitted.

You and I are just autocomplete machines.

I’m surprised ur the first comment I e seen where a human admits humans aren’t that complex in the big scheme of things

If you happened to teach a kid to read right after LLMs got good, it jolts you into thinking that humans are very similar to LLMs. They continually say word X when the page shows word Y, but where word X would make perfect sense, ie semantically similar words are being spat out.

They also make up stories where the words sort of make sense, but it's kinda nonsense. "Hallucinations" all over the place.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#162

Post by Ferret7446 » Tue, Aug 13, 2024, 10:07 AM UTC

skybrian wrote: post #27

It’s “unexpected” because their early training didn’t get rid of it as well as they hoped. LLM’s are good at detecting patterns and like to continue the pattern. They’re starting with autocomplete for voice and training it to do something else. For now, it’s fairly harmless since it’s only a blooper in a lab, but there will likely be open-weights versions of this sort of thing eventually. And there will probably be p…

It's functionally very similar to when, e.g., a child keeps mimicking a character from their favorite cartoon.

Pretty much all of the "quirks" that LLMs exhibit are similar to "human" quirks. Which is great if we're trying to create an AI that makes mistakes like the average human.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#163

Post by peddling-brink » Tue, Aug 13, 2024, 2:03 PM UTC

klyrs wrote: post #15

peddling-brink wrote: post #6

So Open AI has the capability to deep fake any of its users that use the voice chat capability? Yikes.

Did anybody with a passing familiarity with the technology not know this? There's a reason people don't trust their IP to public chat assistants, ya know. It's because they're remorselessly siphoning our data for training purposes. And you've agreed to let them use your inputs for anything they please.

I was under the impression that multiple separate models were in use. I assumed it was something like whisper that converted the user’s input to text then an LLM then text to speech.

I hadn’t realized this was all in one model. Wild stuff.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#164

Post by sigmoid10 » Tue, Aug 13, 2024, 2:19 PM UTC

Animats wrote: post #85

sigmoid10 wrote: post #7

This problem appeared during pre-release testing and has since been solved post-generation using an output classifier that verifies responses, according to the system card release. It was predictable that someone would spin this into a black mirror-esque clickbait story.

Had to track through five links to find out what a "system card" is. It's an instance of a "model card", some broad headings to be filled in.[1] • Model Details. • Intended Use. • Factors. Factors could include demographic or phenotypic groups, environmental conditions, technical attributes, or others • Metrics. • Evaluation Data. • Training Data • Quantitative Analyses • Caveats and Recommendations Meta's PR describ…

>It doesn't, interestingly, seem to include the initial built-in prompts used as constraints.

That's because it is mostly a ploy by profit-driven corporations to allow their researchers to publish some stuff without having them actually reveal anything of value to competitors. Don't be surprised if you find it severely lacking for actual insight.