321–330 of 399 posts
Earlier quoted context omitted.
No, you actually can't. Humans existed for 10s to 100s of thousands of years without text. or even words for that matter.
I disagree: it is language that makes us human.
Earlier quoted context omitted.
> LLMs all Sounds like you don't know how RLHF works. Everything you describe is post-training. Base models can't even chat, they have to be trained to even do basic conversational turn taking.
> Everything you describe is post-training. Base models can't even chat, they have to be trained to even do basic conversational turn taking. So, that's still training then, so not 'post-training'. Just a different training phase.
Earlier quoted context omitted.
LLMs all behave as if they are semi-competent (yet eager, ambitious, and career-minded) interns or administrative assistants, working for a powerful CEO-founder. All sycophancy, confidence and positive energy. "You're absolutely right!" "Here's the answer you are looking for!" "Let me do that for you immediately!" "Here is everything I know about what you just mentioned." Never admitting a mistake unless you directly…
> "You're absolutely right!" "Here's the answer you are looking for!" "Let me do that for you immediately!" "Here is everything I know about what you just mentioned." Never admitting a mistake unless you directly point it out, and then all sorry-this and apologize-that and "here's the actual answer!" It's exactly the kind of personality you always see bubbling up into the orbit of a rich and powerful tech CEO. You ma…
Or that the individual developers see in themselves. Every team I've worked with in my career had one or two of these guys: When the Director or VP came in to town, they'd instantly launch into brown-nose mode. One guy was overt about it and would say things like "So-and-so is visiting the office tomorrow--time to do some petting!" Both the executive and the subordinate have normalized the "royal treatment" on the giving and receiving end.
Earlier quoted context omitted.
> "You're absolutely right!" "Here's the answer you are looking for!" "Let me do that for you immediately!" "Here is everything I know about what you just mentioned." Never admitting a mistake unless you directly point it out, and then all sorry-this and apologize-that and "here's the actual answer!" It's exactly the kind of personality you always see bubbling up into the orbit of a rich and powerful tech CEO. You ma…
An alternative is that these patterns just increase the likelihood of the next thing it outputs being correct, thus are useful to insert during training as the first thing the model says before giving an answer
Earlier quoted context omitted.
You're right, mostly, but the fact remains that the behavior we see is produced by training, and the training is driven by companies run by execs who like this kind of sycophancy. So it's certainly a factor. Humans are producing them, humans are deciding when the new model is good enough for release.
Do you honestly think an executive wanted a chat bot that confidently lies?
1: https://en.wikipedia.org/wiki/The_Emperor%27s_New_Clothes
The thing that bothers me the most about LLMs is how they never seem to understand "the flow" of an actual conversation between humans. When I ask a person something, I expect them to give me a short reply which includes another question/asks for details/clarification. A conversation is thus an ongoing "dance" where the questioner and answerer gradually arrive to the same shared meaning. LLMs don't do this. Instead,…
Earlier quoted context omitted.
I disagree: it is language that makes us human.
I disagree. You're still human if you're deaf and mute. Our intellectual processing powers, or of animals for that matter, has nothing to do with language.
Earlier quoted context omitted.
> Reflect a moment over the fact that LLMs currently are just text generators. You could say the same thing about humans.
How do you reconcile this belief with the fact that we evolved from organisms that had no concept of text?
Earlier quoted context omitted.
A lot of this, I suspect, on the basis of having worked on a supervised fine-tuning project for one of the largest companies in this space, is that providers have invested a lot of money in fine-tuning datasets that sound this way. On the project I did work on, reviewers were not allowed to e.g. answer that they didn't know - they had to provide an answer to every prompt provided. And so when auditing responses, a lo…
Its not the training/tuning, its pretty much the nature of llms. The whole idea is to give a best quess of the token. The more complex dynamics behind the meaning of the words and how those words relate to real world concepts isn't learned.
And I literally saw the effect of this first hand, in seeing how the project I worked on was actively part of training this behaviour into a major model.
As for your assertion they don't learn the more complex dynamics, that was trite and not true already several years ago.