Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT unexpectedly began speaking in a user's cloned voice during testing

arstechnica.com

Post reply on HN
Print view View original

121–130 of 164 posts

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#121

Post by skybrian »

ttul wrote: post #80
lolinder wrote: post #72

Earlier quoted context omitted.

> LLM’s are good at detecting patterns and like to continue the pattern. They’re starting with autocomplete for voice and training it to do something else. This is a great summary of almost everything that goes wrong with LLM applications. LLMs are autocomplete machines, which is why GitHub Copilot is still the most reliably useful application of LLM tech out there. The further you get from autocomplete, the less rel…

I'd respectfully disagree with this characterization of LLMs. While they certainly excel at pattern recognition, calling them mere "autocomplete machines" vastly undersells their capabilities. LLMs demonstrate complex reasoning, multi-modal understanding, and emergent behaviors that go well beyond simple pattern continuation. They've succeeded in areas like mathematical problem-solving, creative tasks, and various re…

I wouldn't say they're mere autocomplete machines, but their tendency to go into a loop to complete a pattern does break the magic, since it's a rather machine-like thing to do.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#122

Post by [deleted] »

mjburgess wrote: post #19
mitthrowaway2 wrote: post #16

Earlier quoted context omitted.

We'll have to gradually get used to the notion that a person's voice, like so many other things we once thought of as intimately personal, is just a coordinate in a high-dimensional vector space.

Those are not contradictions.

[deleted]

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#123

Post by danielmarkbruce »

martin-t wrote: post #105
ttul wrote: post #80

Earlier quoted context omitted.

I'd respectfully disagree with this characterization of LLMs. While they certainly excel at pattern recognition, calling them mere "autocomplete machines" vastly undersells their capabilities. LLMs demonstrate complex reasoning, multi-modal understanding, and emergent behaviors that go well beyond simple pattern continuation. They've succeeded in areas like mathematical problem-solving, creative tasks, and various re…

I don't understand how anybody can still claim LLMs show "complex reasoning". It's been shown time and time again that they'll produce a correct chain of reasoning when given a problem (e.g. wolf, goat, cabbage crossing a river; 3 guards and a door; etc.) that is roughly similar to what's in the training data but will fail when given a sufficiently novel modification _while still producing output that is confidently…

>there was simply text that is statistically likely to be arranged in that way

This is wrong. An LLM can produce text that has never been arranged that way in training data.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#124

Post by danielmarkbruce »

ericlewis wrote: post #106
martin-t wrote: post #105

Earlier quoted context omitted.

I don't understand how anybody can still claim LLMs show "complex reasoning". It's been shown time and time again that they'll produce a correct chain of reasoning when given a problem (e.g. wolf, goat, cabbage crossing a river; 3 guards and a door; etc.) that is roughly similar to what's in the training data but will fail when given a sufficiently novel modification _while still producing output that is confidently…

Perhaps it’s because I know human beings that have the exact same operation and failure mode as the LLM here and I’m probably not the only one. Failing at something you’ve never seen and faking through it is a very human endeavor.

If you teach your kids to read, they behave almost exactly like an LLM in so many cases it's eery.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#125

Post by danielmarkbruce »

lolinder wrote: post #118
wyager wrote: post #115

Earlier quoted context omitted.

> all problems that we want to feed to an LLM therefore must be translated to autocomplete. I don't disagree with this, but I do disagree with this earlier statement: > The further you get from autocomplete, the less reliable the resulting product Any naturally sequential problem is trivial to translate to autocomplete with minimal loss of fidelity.

In other words, would it be fair to say that any naturally sequential problem is not very far from autocomplete? Again, I think you're putting words in my mouth and thoughts in my head that aren't there. A lot of people have reacted to AI hype by going the other way and underestimating them—that's not me. I think there are lots of problems they can solve, I just think they all boil down to autocomplete and if you can…

People fine tune LLMs for classification tasks.

This is completely wrong.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#126

Post by mike_hearn »

lolinder wrote: post #118
wyager wrote: post #115

Earlier quoted context omitted.

> all problems that we want to feed to an LLM therefore must be translated to autocomplete. I don't disagree with this, but I do disagree with this earlier statement: > The further you get from autocomplete, the less reliable the resulting product Any naturally sequential problem is trivial to translate to autocomplete with minimal loss of fidelity.

In other words, would it be fair to say that any naturally sequential problem is not very far from autocomplete? Again, I think you're putting words in my mouth and thoughts in my head that aren't there. A lot of people have reacted to AI hype by going the other way and underestimating them—that's not me. I think there are lots of problems they can solve, I just think they all boil down to autocomplete and if you can…

This is kind of reductionist. It's like saying that a human writing a book is just doing manual word completion starting from the title. It's technically correct, but what insight is contributed? Would anything about this conversation be different if someone trained a model that did diffusion-like inference in which every possible word in the answer is pushed towards the final result simultaneously? Probably not.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#127

Post by danielmarkbruce »

samtho wrote: post #120
wyager wrote: post #112

Earlier quoted context omitted.

> We know how LLMs work fundamentally We know how they work only at the lowest level (the arithmetic operations) and the highest level (the optimization criterion and the representation of various layers, like the input/output layer and for things we can easily probe like embedding matrices). We do not know "what they are doing" on the inner layers. This is an area of active research. > They do not have the ability t…

You talk about about as if a human-created neural network is at the same level as quantum physics where there are limits as to our understanding. We know very well how large language models work even if the capabilities of this technology are actively being explored. You along with others here are far overstating the unknowns we have within the context of AI, whether this is the result of a misinformation campaign ta…

For the definition of "understand" that most people use, humans don't understand things which are highly complex. We don't really understand the weather, we can't predict it, it's too complex. But, you can break it down into matter and forces and energy and simulate it and get pretty darn good predictions. We can now throw it in a deep learning model and get good predictions. But, to suggest we "understand it" doesn't gel with most people's definition of "understand".

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#128

Post by mike_hearn »

samtho wrote: post #120
wyager wrote: post #112

Earlier quoted context omitted.

> We know how LLMs work fundamentally We know how they work only at the lowest level (the arithmetic operations) and the highest level (the optimization criterion and the representation of various layers, like the input/output layer and for things we can easily probe like embedding matrices). We do not know "what they are doing" on the inner layers. This is an area of active research. > They do not have the ability t…

You talk about about as if a human-created neural network is at the same level as quantum physics where there are limits as to our understanding. We know very well how large language models work even if the capabilities of this technology are actively being explored. You along with others here are far overstating the unknowns we have within the context of AI, whether this is the result of a misinformation campaign ta…

wyager is correct. There are very serious limits to our understanding of what's going on inside these networks. Even how an LLM answers simple factual questions like "The capital of France is ..." is only now just coming into view. And the moment it gets more complex than that, interpretability is lost again.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#129

Post by mike_hearn »

cdrini wrote: post #90
c0mbonat0r wrote: post #38

Earlier quoted context omitted.

Can someone pleasee convince why i shouldn't be absolutely shit out of my mind cynical about this innovation? we are literally seeing the downfall of trust in society. and no, i dont believe i am exaggerating

I think the other comments make a good argument about how other forms of technology have also degraded trust, but that we've found a way through. I'll also add that I think one potential way we could reinstate trust is through signed multimedia. Cameras/microphones/etc could sign the videos/audio they create in a way that can be used to verify that the media hasn't been doctored. Not sure if that's actually a feasibl…

It's feasible with advanced enough tech. The hard part isn't getting cameras to sign the files they produce. The hard part is to preserve the chain of custody as images are cropped, rescaled, recompressed etc. You can do it with tech like Intel SGX. But you also need serious defense of the camera platforms against hacking, of the CPUs, of the software stacks. And there's no demand. News orgs feel they should be implicitly trusted due to their brands, so why would they use complicated tech to build trust?

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#130

Post by bane »

martin-t wrote: post #105
ttul wrote: post #80

Earlier quoted context omitted.

I'd respectfully disagree with this characterization of LLMs. While they certainly excel at pattern recognition, calling them mere "autocomplete machines" vastly undersells their capabilities. LLMs demonstrate complex reasoning, multi-modal understanding, and emergent behaviors that go well beyond simple pattern continuation. They've succeeded in areas like mathematical problem-solving, creative tasks, and various re…

I don't understand how anybody can still claim LLMs show "complex reasoning". It's been shown time and time again that they'll produce a correct chain of reasoning when given a problem (e.g. wolf, goat, cabbage crossing a river; 3 guards and a door; etc.) that is roughly similar to what's in the training data but will fail when given a sufficiently novel modification _while still producing output that is confidently…

I agree that LLMs don't really reason, but I'm starting to think that the name "Large Language Model" is just wrong enough to create this confusion. What they seem to be doing is modeling human reasoning as encoded in language, and then enable some mixing and matching of elements of that encoded reasoning.

This somewhat means that these models are trapped within the universe of "human capable reasoning" with some possibility of escaping it through the stochastic generative processes they're built on. But they simply can't think through novel problems and arrive at new conclusions.

Furthermore, they're limited by the fact they're built on human knowledge as encoded in text, which terribly imprecise and fluid. Any hope to have a path to AGI, where reasoning might actually happen, will have to have something far more rigorous for the internal reasoning, with language just being a clever interface rather than the mechanism by which the thinking is done in.

They're really "Large Analogy Engines" or maybe "Large Captured Reasoning Engines" and are an incredible technology.

Post reply on HN