Reasoning models don't always say what they think

Perhaps I misunderstand your point but it seems to me that by the same logic a simple gradient descent algorithm wired up to a variety of different models and simulations would qualify as generalization during the training phase.

The trouble with this is that it only ever "generalizes" approximately as far as the person configuring the training run (and implementing the simulation and etc) ensures that it happens. In which case it seems analogous to an explicitly programmed algorithm to me.

Even if we were to accept the training phase as a very limited form of generalization it still wouldn't apply to the output of that process. The trained LLM as used for inference is no longer "learning".

The point I was trying to make with the chess engine was that it doesn't seem that generalization is required in order to perform that class of tasks (at least in isolation, ie post-training). Therefore, it should follow that we can't use "ability to perform the task" (ie beat a human at that type of board game) as a measure for whether or not generalization is occurring.

Hypothetically, if you could explain a novel rule set to a model in natural language, play a series of several games against it, and following that it could reliably beat humans at that game, that would indeed be a type of generalization. However my next objection would then be, sure, it can learn a new turn based board game, but if I explain these other five tasks to it that aren't board games and vary widely can it also learn all of those in the same way? Because that's really what we seem to mean when we say that humans or dogs or dolphins or whatever possess intelligence in a general sense.

I thought the definition of literate was "can interpret text in place of the spoken word". At which point it's worth noting that text is a much lower bandwidth channel than in person communication. Also worth noting that, ex, a mute person could still be considered intelligent.

Is it necessarily the case that you could discern general intelligence via a test with fixed structure, known to all parties in advance, carried out via a synthesized monotone voice? I'm not saying "you definitely can't do that" just that I don't see why we should a priori assume it to be possible.

Now that likely seems largely irrelevant and out in the weeds and normally I would feel that way. But if you're going to suppose that we can't cleanly differentiate LLMs from humans then it becomes important to ask if that's a consequence of the LLMs actually exhibiting what we would consider general intelligence versus an inherent limitation of the modality in which the interactions are taking place.

Personally I think it's far more likely that we just don't have very good tests yet, that our working definition of "general intelligence" (as well as just "intelligence") isn't all that great yet, and that in the end many humans who we consider to exhibit a reasonable level of such will nonetheless fail to pass tests that are based solely on an isolated exchange of natural language.

Shifting goalposts usually (at least in my understanding) refers to changing something without valid justification that was explicitly set in a previous step (subjective wording I realize - this is off the top of my head). In an adversarial context it would be someone attempting to gain an advantage by subtly changing a premise in order to manipulate the conclusion.

An incomplete list, in contrast, is not a full set of goalposts. It is more akin to a declared lower bound.

I also don't think it to applies to the case where the parties are made aware of a change in circumstances and update their views accordingly.

> You feel that LLMs are not intelligent. I feel that they may be intelligent.

Weirdly enough I almost agree with you. LLMs have certainly challenged my notion of what intelligence is. At this point I think it's more a discussion of what sorts of things people are referring to when they use that word and if we can figure out an objective description that distinguishes those things from everything else.

> Without a decent definition of what intelligence is, the entire argument is silly.

I completely agree. My only objection is to the notion that goalposts have been shifted since in my view they were never established in the first place.

Humans think they can produce chain-of-reasoing, but it has been shown many times (and is self evident if you pay attention) that your brain is making decisions before you are aware of it.

If I ask you to think of a movie, go ahead, think of one.....whatever movie just came into your mind was not picked by you, it was served up to you from an abyss.

This is a discussion of semantics. First I spent much of my career in high end quant finance and what we are doing today is night and day different in terms of the generality and effectiveness. Second, almost all the hallmarks of AI I carried with me prior to 2001 have more or less been ticked off - general natural language semantically aware parsing and human like responses, ability to process abstract concepts, reason abductively, synthesize complex concepts. The fact it’s not aware - which it’s absolutely is not - does not make it not -intelligent-.

The thing people latch onto is modern LLM’s inability to reliably reason deductively or solve complex logical problems. However this isn’t a sign of human intelligence as these are learned not innate skills, and even the most “intelligent” humans struggle at being reliable at these skills. In fact classical AI techniques are often quite good at these things already and I don’t find improvements there world changing. What I find is unique about human intelligence is its abductive ability to reason in ambiguous spaces with error at times but with success at most others. This is something LLMs actually demonstrate with a remarkably human like intelligence. This is earth shattering and science fiction material. I find all the poopoo’ing and goal post shifting disheartening.

What they don’t have is awareness. Awareness is something we don’t understand about ourselves. We have examined our intelligence for thousands of years and some philosophies like Buddhism scratch the surface of understanding awareness. I find it much less likely we can achieve AGI without understanding awareness and implementing some proximate model of it that guides the multi modal models and agents we are working on now.