251–260 of 279 posts
The fact that it was ever seriously entertained that a "chain of thought" was giving some kind of insight into the internal processes of an LLM bespeaks the lack of rigor in this field. The words that are coming out of the model are generated to optimize for RLHF and closeness to the training data, that's it! They aren't references to internal concepts, the model is not aware that it's doing anything so how could it…
Earlier quoted context omitted.
I frequently see this characterization and can't agree with it. If I say "well I suppose you'd at least need to do A to qualify" and then later say "huh I guess A wasn't sufficient, looks like you'll also need B" that is not shifting the goalposts. At worst it's an incomplete and ad hoc specification. More realistically it was never more than an educated guess to begin with, about something that didn't exist at the t…
> If I say "well I suppose you'd at least need to do A to qualify" and then later say "huh I guess A wasn't sufficient, looks like you'll also need B" that is not shifting the goalposts. I dunno, that seems like a pretty good distillation of what moving the goalposts is. > I’ll know it when I see it, and I haven’t seen it. QED While pithily put, thats not a compelling argument. You feel that LLMs are not intelligent.…
Only if you don't understand what "the goalposts" means. The goalpost isn't "pass the turing test", the goalpost is "manage to do all the same kind of intellectual tasks that humans are", nobody has moved that since the start in the quest for AI.
The fact that it was ever seriously entertained that a "chain of thought" was giving some kind of insight into the internal processes of an LLM bespeaks the lack of rigor in this field. The words that are coming out of the model are generated to optimize for RLHF and closeness to the training data, that's it! They aren't references to internal concepts, the model is not aware that it's doing anything so how could it…
Earlier quoted context omitted.
> Generalization is the ability for a model to perform well on new unseen data within the same task that it was trained for. By that logic a chess engine can generalize in the same way that AlphaGo Zero does. It is a black box that has never seen the vast majority of possible board positions. In fact it's never seen anything at all because unlike an ML model it isn't the result of an optimization algorithm (at least…
You're way off. This is not my personal definition of generalization. We are talking about a very specific technical term in the context of machine learning. An explicitly programmed chess engine does not generalize, by definition. It doesn't learn from data. It is an explicitly programmed algorithm. I recommend you go do some reading about machine learning basics. https://www.cs.toronto.edu/~lczhang/321/notes/notes0…
As far as metrics of intelligence go, the algorithm is a black box. We don't care how it works or how it was constructed. The only thing we care about is (something like) how well it performs across an array of varied tasks that it hasn't encountered before. That is to say, how general the black box is.
Notice that in the case of typical ML algorithms the two usages are equivalent. If the approach generalizes (from training) then the resulting black box would necessarily be assessed as similarly general.
So going back up the thread a ways. Someone quotes Chollet as saying that LLMs can't generalize. You object that he sets the bar too high - that, for example, they generalize just fine at Go. You can interpret that using either definition. The result is the same.
As far as measuring intelligence is concerned, how is "generalizes on the task of Go" meaningfully better than a procedural chess engine? If you reject the procedural chess engine as "not intelligent" then it seems to me that you must also reject an ML model that does nothing but play Go.
> An explicitly programmed chess engine does not generalize, by definition. It doesn't learn from data. It is an explicitly programmed algorithm.
Following from above, I don't see the purpose of drawing this distinction in context since the end result is the same. Sure, without a training task you can't compare performance between the training run and something else. You could use that as a basis to exclude entire classes of algorithms, but to what end?
Earlier quoted context omitted.
The LLM's output is chaotic relative to the input, but it's deterministic right? Same settings, same model, same input, .. same output? Where does the chain get broken here?
Depends on what you mean specifically by the output. The actual neural network will produce deterministic outputs that could be interpreted as probability values for various tokens. But the interface you'll commonly see used in front of these models will then non-deterministiclaly choose a single next token to output based on those probabilities. Then, this single randomly chosen output is fed back into the network t…
Earlier quoted context omitted.
What's wrong with just calling them smart algorithmic models? Being smart allows somewhat to be wrong, as long as that leads to a satisfying solution. Being intelligent on the other hand requires foundational correctness in concepts that aren't even defined yet. EDIT: I also somewhat like the term imperative knowledge (models) [0] [0]: https://en.wikipedia.org/wiki/Procedural_knowledge
The problem with "smart" is that they fail at things that dumb people succeed at. They have ludicrous levels of knowledge and a jaw dropping ability to connect pieces while missing what's right in front of them. The gap makes me uncomfortable with the implications of the word "smart". It is orthogonal to that.
Funnily enough, you can also observe that in humans. The number of times I have observed people from highly intellectual, high income/academic families struggle with simple tasks that even the dumbest people do with ease is staggering. If you're not trained for something and suddenly confronted with it for the first time, you will also in all likelihood fail. "Smart" is just as ill-defined as any other clumsy approach to define intelligence.
Earlier quoted context omitted.
>internal concepts, the model is not aware that it's doing anything so how could it "explain itself" This in a nutshell is why I hate that all this stuff is being labeled as AI. Its advanced machine learning (another term that also feels inaccurate but I concede is at least closer to whats happening conceptually) Really, LLMs and the like still lack any model of intelligence. Its, in the most basic of terms, algorith…
While I agree that LLMs are hardly sapient, it's very hard to make this argument without being able to pinpoint what a model of intelligence actually is. "Human brains lack any model of intelligence. It's just neurons firing in complicated patterns in response to inputs based on what statistically leads to reproductive success"
The fact that it was ever seriously entertained that a "chain of thought" was giving some kind of insight into the internal processes of an LLM bespeaks the lack of rigor in this field. The words that are coming out of the model are generated to optimize for RLHF and closeness to the training data, that's it! They aren't references to internal concepts, the model is not aware that it's doing anything so how could it…
But we have no direct insight into most of our internal thought processes. And we have direct experimental data showing our brain will readily make up bullshit about our internal thought processes (split brain experiments, where one brain half is asked to justify a decision made that it didn't make; it will readily make claims about why it made the decision it didn't make)
Earlier quoted context omitted.
Humans think they can produce chain-of-reasoing, but it has been shown many times (and is self evident if you pay attention) that your brain is making decisions before you are aware of it. If I ask you to think of a movie, go ahead, think of one.....whatever movie just came into your mind was not picked by you, it was served up to you from an abyss.
How is that in conflict with the fact that humans can introspect?
We're doing the equivalent of LLM's and making up a plausible explanation for how we came to a conclusion, not reflecting reality.
Earlier quoted context omitted.
> it all rests on (relatively) simple mathematics. We know this is true. We also know that means it has limitations and can't actually reason information. What do you imagine is happening inside biological minds that enables reasoning that is something different to, a lot of, "simple mathematics"? You state that because it is built up of simple mathematics it cannot be reasoning, but this does not follow at all, unle…
Because whats inside our minds is more than mathematics, or we would be able to explain human behavior with the purity of mathematics, and so far, we can't. We can prove the behavior of LLMs with mathematics, because its foundations are constructed. That also means it has the same limits of anything else we use applied mathematics for. Is the broad market analysis that HFT firms use software for to make automated tra…
For your claim to be true, it would need to be provably impossible to explain human behavior with mathematics.
For that to be true, humans would need to be able to compute functions that are computable but outside the Turing computable, outside the set of lambda functions, and outside the set of generally recursive functions (the tree are computationally equivalent).
We know of no such function. We don't know how to construct such a function. We don't know how it would be possible to model such a function with known physics.
It's an extraordinary claim, with no evidence behind it.
The only evidence needed would be a single example of a function we can compute outside the Turing computable set, which would seem to make the lack of such evidence make it rather improbably.
It could still be true, just like there could truly be a teapot in orbit between Earth and Mars. I'm nt holding my breath.