241–250 of 279 posts
Earlier quoted context omitted.
The LLM's output is chaotic relative to the input, but it's deterministic right? Same settings, same model, same input, .. same output? Where does the chain get broken here?
Now compare a human to an LSTM with persistent internal state that you can't reset.
The fact that it was ever seriously entertained that a "chain of thought" was giving some kind of insight into the internal processes of an LLM bespeaks the lack of rigor in this field. The words that are coming out of the model are generated to optimize for RLHF and closeness to the training data, that's it! They aren't references to internal concepts, the model is not aware that it's doing anything so how could it…
so if you deterministically replay an inference session n times on a single question, and each time in the middle you subtly change the context buffer without changing its meaning, does it impact the likelihood or path of getting to the correct solution in a meaningful way?
Earlier quoted context omitted.
> The fact that you can reason about intelligence is a counter argument to this The fact that we can provide a chain of reasoning, and we can think that it is about intelligence, doesn't mean that we were actually reasoning about intelligence. This is immediately obvious when we encounter people whose conclusions are being thrown off by well-known cognitive biases, like cognitive dissonance. They have no trouble prod…
Humans think they can produce chain-of-reasoing, but it has been shown many times (and is self evident if you pay attention) that your brain is making decisions before you are aware of it. If I ask you to think of a movie, go ahead, think of one.....whatever movie just came into your mind was not picked by you, it was served up to you from an abyss.
Earlier quoted context omitted.
You're muddling up some technical concepts here in a very confusing way. Generalization is the ability for a model to perform well on new unseen data within the same task that it was trained for. It's not about the training process itself. Suppose I showed you some examples of multiplication tables, and you figured out how to multiply 19 * 42 without ever having seen that example before. That is generalization. You h…
> Generalization is the ability for a model to perform well on new unseen data within the same task that it was trained for. By that logic a chess engine can generalize in the same way that AlphaGo Zero does. It is a black box that has never seen the vast majority of possible board positions. In fact it's never seen anything at all because unlike an ML model it isn't the result of an optimization algorithm (at least…
We are talking about a very specific technical term in the context of machine learning.
An explicitly programmed chess engine does not generalize, by definition. It doesn't learn from data. It is an explicitly programmed algorithm.
I recommend you go do some reading about machine learning basics.
Earlier quoted context omitted.
Seems pretty reasonable to presuppose this when you filter to people who are literate. That’s darn near a definition of literate, that you can engage with the text intelligently.
I thought the definition of literate was "can interpret text in place of the spoken word". At which point it's worth noting that text is a much lower bandwidth channel than in person communication. Also worth noting that, ex, a mute person could still be considered intelligent. Is it necessarily the case that you could discern general intelligence via a test with fixed structure, known to all parties in advance, carr…
Note that functional illiteracy is not some niche phenomenon, it's a huge problem in many school systems. In my own country (Romania), while the rate of illiteracy is something like <1% of the populace, the rate of functional illiteracy is estimated to be as high as 45% of those finishing school.
Earlier quoted context omitted.
>internal concepts, the model is not aware that it's doing anything so how could it "explain itself" This in a nutshell is why I hate that all this stuff is being labeled as AI. Its advanced machine learning (another term that also feels inaccurate but I concede is at least closer to whats happening conceptually) Really, LLMs and the like still lack any model of intelligence. Its, in the most basic of terms, algorith…
While I agree that LLMs are hardly sapient, it's very hard to make this argument without being able to pinpoint what a model of intelligence actually is. "Human brains lack any model of intelligence. It's just neurons firing in complicated patterns in response to inputs based on what statistically leads to reproductive success"
Are you sure about that ? Do we have proof of that ? In happened all the time trought history of science that a lot of scientists were convinced of something and a model of reality up until someone discovers a new proof and or propose a new coherent model. That’s literally the history of science, disprove what we thought was an established model
Earlier quoted context omitted.
datasets and search engines are deterministic. humans, and llms are not.
The LLM's output is chaotic relative to the input, but it's deterministic right? Same settings, same model, same input, .. same output? Where does the chain get broken here?
I would ultimately call the result non-deterministic. You could make it deterministic relatively easily by having a deterministic process for choosing a single token from all of the outputs of the NN (say, always pick the one with the highest weight, and if there are multiple with the same weight, pick the first one in token index order), but no one normally does this, because the results aren't that great per my understanding.
Earlier quoted context omitted.
"There exists a generally accepted baseline definition for what crosses the threshold of intelligent behavior" not really. The whole point they are trying to make is that the capability of these models IS ALREADY muddying the definition of intelligence. We can't really test it because the distribution its learned is so vast. Hence why he have things like ARC now. Even if its just gradient descent based distribution l…
Peoples’ memories are so short. Ten years ago the “well accepted definition of intelligence” was whether something could pass the Turing test. Now that goalpost has been completely blown out of the water and people are scrabbling to come up with a new one that precludes LLMs. A useful definition of intelligence needs to be measurable, based on inputs/outputs, not internal state. Otherwise you run the risk of dictatin…
Various chat bots have long been able to pass more limited versions of a Touring test. The most extreme constraint allows for simply replaying a canned conversation which with a helpful human assistant makes it indistinguishable from a human. But exploiting limitations on a testing format doesn’t have anything to do with testing for intelligence.
> There’s no specific reason why the reported Chain-of-Thought must accurately reflect the true reasoning process; Isn't the whole reason for chain-of-thought that the tokens sort of are the reasoning process? Yes, there is more internal state in the model's hidden layers while it predicts the next token - but that information is gone at the end of that prediction pass. The information that is kept "between one token…
What I think is interesting about this is that for the most part reading the reasoning output is something we can understand. The tokens as produced form english sentences, make intuitive sense. If we think of the reasoning output block as basically just "hidden state" then one could imagine that a there might be a more efficient representation that trades human understanding for just priming the internal state of the model.
In some abstract sense you can already get that by asking the model to operate in different languages. My first experience with reasoning models where you could see the output of the thinking block I think was QwQ which just reasoned in Chinese most of the time, even if the final output was German. Deepseek will sometimes keep reasoning in English even if you ask it German stuff, sometimes it does reason in German. All in all, there might be a more efficient representation of the internal state if one forgoes human readable output.