> If this is what you mean by "reasoning by analogy" then I hate to tell you this, but "reasoning by analogy" is "reasoning" in itself. There's really no form of reasoning beyond associating things you already know. Think about it.
What's special about humans is that we can obtain an understanding of what chains of associations to make and when, to achieve the goal at hand, even without being told which method to use. We know when to do arithmetic, trace a program, decipher someone else's thoughts, etc. Also, we know to resort to a fallback method if the current one isn't working. We can assist models with this process in the special case (e.g., that tool-using model), but I suspect the general case will remain elusive for a while yet.
That is to say, I'll grant you that associations can act as a primitive operation of intelligence, much as metal cylinders and flames are primitive parts of a rocket, but I suspect that making a LLM "generally intelligent" or "sentient" will be far harder still.
> The other thing is, I feel it knows math as well as some D student in highschool. Are you saying the D student in highschool can't understand anything? No. So you really can't use this logic to dismiss LLMs because PLENTY of people don't know math well either, and you'd have to dismiss them as sentient beings if you followed your own reasoning to the logical conclusion.
I was just using that as a specific example of the general issue: it doesn't notice that its answer is wrong and its particular method can never work, and it refuses to try a meaningfully different method (no matter how much I prompt it to). Its immediate mistakes might look similar to those of a poor student, but I suspect they come from a different underlying problem. (After all, the student has seen perhaps a thousand algebra problems at most, whereas the model has seen millions and millions. Also, the student often )
> What's impossible here is to flip your bias. You and others like you will still be naysaying LLMs even after they take your job.
You have me wrong: I'm not saying that augmenting LLMs can't make them reliable enough to take over some people's jobs. But I am disputing that LLMs alone will produce AGIs capable of outwitting any human, taking over the world, advancing the limits of math and science, or many of those other grandiose claims.
Anyway, I'm not trying to be particularly stubborn about this like some people are; I'm keeping a close eye on the space. But I'll only believe it when I see it (and no later), and I don't think I've quite seen it yet.