Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT unexpectedly began speaking in a user's cloned voice during testing

arstechnica.com

Post reply on HN
Print view View original

151–160 of 164 posts

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#151

Post by afarviral »

lolinder wrote: post #118
wyager wrote: post #115

Earlier quoted context omitted.

> all problems that we want to feed to an LLM therefore must be translated to autocomplete. I don't disagree with this, but I do disagree with this earlier statement: > The further you get from autocomplete, the less reliable the resulting product Any naturally sequential problem is trivial to translate to autocomplete with minimal loss of fidelity.

In other words, would it be fair to say that any naturally sequential problem is not very far from autocomplete? Again, I think you're putting words in my mouth and thoughts in my head that aren't there. A lot of people have reacted to AI hype by going the other way and underestimating them—that's not me. I think there are lots of problems they can solve, I just think they all boil down to autocomplete and if you can…

What's the difference between "talk" or "produce output" in your mind? I feel that automatic completion makes it sound as if the complete result was already supplied to the model or that it is akin to just the most probable next word from simple frequency rather than the "thinking" that is done by the model to produce the next token. Autocomplete doesn't adequately describe anything. Like imagine a hypothetical machine of infinite ability that still works in a way where it produces tokens in a series from previous tokens, you would still be arguing it is called autocomplete and bothering me.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#152

Post by afarviral »

gls2ro wrote: post #142
wyager wrote: post #78

Earlier quoted context omitted.

You are making a very popular mistake: confusing the training procedure for LLMs (autocomplete) with "how they work"/their internal ontology (mostly unknown). When we teach children how to do arithmetic, we have them predict missing items in equations. We don't accuse them of "only doing autocomplete". The same applies for LLMs.

I am always confused when people compare LLMs with children - there is an obvious evolution of reasoning in children as the time passes by. This can be visible on a time span of months. Using the same analogy: I dont see this in LLMs, I can speak with an LLM months and if we dont release a new version it is the same. So they are not like children and I am not sure we really understand learning in children more so to…

If you child's memory was erased each day I imagine you wouldn't notice any progress, indeed.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#153

Post by afarviral »

samtho wrote: post #97
wyager wrote: post #78

Earlier quoted context omitted.

You are making a very popular mistake: confusing the training procedure for LLMs (autocomplete) with "how they work"/their internal ontology (mostly unknown). When we teach children how to do arithmetic, we have them predict missing items in equations. We don't accuse them of "only doing autocomplete". The same applies for LLMs.

This is only a common mistake to the subset of people who labor under the delusion that we have actually stumbled upon Generalized Artificial Intelligence. We know how LLMs work fundamentally and what their limits are. LLMs are only able to make “correct sounding” statements which have the side effect of being correct a certain percentage of the time. They do not have the ability to reason nor engage in high level th…

How can you say they don't reason if reasoning is required to produce high rates of success that previously have only been successfully answered using reasoning? Its not like they are guessing randomly but just cooindidentally having high rates of accuracy. They clearly have an emergent property of reasoning. Perhaps reasoning is too ill-defined of a word. They are deducing or calculating the answer from their being. What we call the thing that occurs when they produce an insightful answer is less telling than the fact that they produce answers previously only possible by thinking and reasoning humans.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#154

Post by afarviral »

visarga wrote: post #34
mitthrowaway2 wrote: post #16

Earlier quoted context omitted.

We'll have to gradually get used to the notion that a person's voice, like so many other things we once thought of as intimately personal, is just a coordinate in a high-dimensional vector space.

> is just a coordinate in a high-dimensional vector space Everything is, including types of personality, physiognomy and demographics. Anything that can be related to anything else is embeddable.

Is this a well studied topic I can research or just a guess?

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#155

Post by danielmarkbruce »

lolinder wrote: post #148
danielmarkbruce wrote: post #143

Earlier quoted context omitted.

The justification is that people are fine tuning LLMs for classification. They take out the last layer, replace it with a layer which maps to n classes instead of vocab_size, and the training data Y's aren't next word, they are a class label (I have a job which does binary classification, for example) It's just completely wrong to say everything in LLM land is autocomplete. It's trivial and common to do the above.

That's still autocomplete. You use it by feeding in a context (all the text so far) and asking it to produce the next word (your fine-tuned classification token). The only difference is you don't ask for more tokens once you have one. That's a very clever way of reducing a problem to autocomplete, but it doesn't change the paradigm.

If an email says "Respond to win $100 now!" and a classifier has it as 99%/1% for two classes representing spam/not spam, "spam" is not a sensible next token, it's a classification. The model is not trying to predict the next token, it's trying to classify the entire body of text. The training data isn't a bunch of samples where y is whatever came after those tokens.

It's a silly way to think about it. Have you seen how people are fine tuning for classification? It's not like fine tuning for instruction or summarization etc, which are still using next token prediction and where the last layer is still mapping to vocab_size outputs.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#156

Post by consteval »

danielmarkbruce wrote: post #96
lolinder wrote: post #72

Earlier quoted context omitted.

> LLM’s are good at detecting patterns and like to continue the pattern. They’re starting with autocomplete for voice and training it to do something else. This is a great summary of almost everything that goes wrong with LLM applications. LLMs are autocomplete machines, which is why GitHub Copilot is still the most reliably useful application of LLM tech out there. The further you get from autocomplete, the less rel…

You and I are just autocomplete machines.

I disagree, because this comment doesn't feel right to me and therefore it isn't right.

That alone disproves this stance. This higher podium for the human race and even the ability to feel means we are more than just autocomplete machines. We have allegiance, irrationality, animalistic instinct.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#157

Post by consteval »

AmericanChopper wrote: post #60
c0mbonat0r wrote: post #38

Earlier quoted context omitted.

Can someone pleasee convince why i shouldn't be absolutely shit out of my mind cynical about this innovation? we are literally seeing the downfall of trust in society. and no, i dont believe i am exaggerating

The way I see it LLMs are just making it more obvious to see all the flaws with the existing levels of trust. Humans have never had access to universal truths, or universal ways of validating anything. Any claim anybody makes could be intentionally or unintentionally deceitful or untrue. The idea that there are sources you can trust to do your thinking for you is the more dangerous illusion in my opinion, and I’m not…

> The idea that there are sources you can trust to do your thinking for you is the more dangerous illusion in my opinion

The difference between economically successful countries like the US and the peripheral countries is we are a high-trust society.

I don't spend 100 hours chemically testing my food because I have faith it is safe to eat. I don't waste money on scam after scam because I have faith most businesses are legitimate. If I'm a business, I can order stuff and more stuff and I trust the spec.

Our outsourcing of that trust to other people is what makes us economically successful.

Other countries which don't have this trust focus on basic tasks. Gathering food, water, shelter, and basic infrastructure. Because ultimately every man is out for himself. They aren't building software and airplanes and whatnot. Because as complexity increases, the more people are involved and therefore the most trust is required. Trust is required because of the fundamental limitations of human meat space - we have limited time and survival needs.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#158

Post by danielmarkbruce »

consteval wrote: post #156
danielmarkbruce wrote: post #96

Earlier quoted context omitted.

You and I are just autocomplete machines.

I disagree, because this comment doesn't feel right to me and therefore it isn't right. That alone disproves this stance. This higher podium for the human race and even the ability to feel means we are more than just autocomplete machines. We have allegiance, irrationality, animalistic instinct.

It was tongue in cheek.

But seriously, I could push back and say: all you did is put one idea after another there. If you spend time observing your thought, an enormous amount of it is just putting one idea after another.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#159

Post by consteval »

danielmarkbruce wrote: post #158
consteval wrote: post #156

Earlier quoted context omitted.

I disagree, because this comment doesn't feel right to me and therefore it isn't right. That alone disproves this stance. This higher podium for the human race and even the ability to feel means we are more than just autocomplete machines. We have allegiance, irrationality, animalistic instinct.

It was tongue in cheek. But seriously, I could push back and say: all you did is put one idea after another there. If you spend time observing your thought, an enormous amount of it is just putting one idea after another.

This implies my thoughts are even observable. I don't know that they are. I mean, my brain is doing the thinking and it's also doing the observing. If it wanted to, it could keep things from me.

Re: ChatGPT unexpectedly began speaking in a user's cloned voice during testing

#160

Post by danielmarkbruce »

consteval wrote: post #159
danielmarkbruce wrote: post #158

Earlier quoted context omitted.

It was tongue in cheek. But seriously, I could push back and say: all you did is put one idea after another there. If you spend time observing your thought, an enormous amount of it is just putting one idea after another.

This implies my thoughts are even observable. I don't know that they are. I mean, my brain is doing the thinking and it's also doing the observing. If it wanted to, it could keep things from me.

Yup, all fair points. Like I said, the comment was mostly tongue in cheek.
Post reply on HN