Andrej Karpathy – It will take a decade to work through the issues with agents

I think there are significantly more than 2, when you start to count variations through the spectrum of neurodiversity.

Spatial thinkers, for example, or the hyperlexic.

Meaning for hyperlexics is more akin to finding meaning in the edges of the graph, rather than the vertices. The form of language contributing a completely separate graph of knowledge, alongside its content, creating a rich, multimodal form of understanding.

Spatial thinkers have difficulty with procedural thinking, which is how most people are taught. Rather than the series of steps to solve the problem, they see the shape of the transform. LLMs as an assistive device can be very useful for spatial thinkers in providing the translation layer between the modes of thought.

  > Meanwhile humans don't need any data at all

I don't agree with this and I don't think any biologist or neuroscientist would either.

1) Certainly the data I discussed exists. No creature comes out a blank slate. I'll be bold enough to say that this is true even for viruses, even if we don't consider them alive. Automata doesn't mean void of data and I'm not sure why you'd ascribe this to life or humans.

2) humans are processing data from birth (technically before too but that's not necessary for this conversation and I think we all know that's a great way to have an argument and not address our current conversation). This is clearly some active/online/continual/ reinforcement/wherever-word-you-want-to-use learning.

It's weird to suggest an either or situation. All evidence points to "both". Looking at different animals even see both but also with different distributions.

I think it's easy to over simplify the problem and the average conversation tends to do this. It's clearly a complex with many variables at play. We can't approximate with any reasonable accuracy by ignoring or holding them constant. They're coupled.

  > The reward function can't be created by the brain, because it would be circular.

Why not? I'm absolutely certain I can create my own objectives and own metrics. I'm certain my definition of success is different from yours.

  > It would be like giving yourself a high, without even needing drugs

Which is entirely possible. Maybe it takes extreme training to do extreme versions but it's also not like chemicals like dopamine are constant. You definitely get a rush by completing goals. People become addicted to things like videogames, high risk activities like sky diving, or even arguing on the internet.

Just because there are externally driven or influenced goals doesn't mean internal ones can't exist. Our emotions can be driven both externally and internally.

  > Notice something?

You're using too simple of a model. If you use this model then the solution is as easy as giving a robot self preservation (even if we need to wait a few million years). But how would self preservation evolve beyond its initial construction without the ability to metaprocess and refine that goal? So I think this should highlight a major limitation in your belief. As I see it, the only other way is a changing environment that somehow allows continued survival by the constructions and precisely evolves such that the original instructions continue to work. Even with vague instructions that's an unstable equilibrium. I think you'll find there's a million edge cases even if it seems obvious at first. Or read some Asimov ;)

AGI is forever away if you stay blinkered to the LLM/difussion stochastic parrot program, which might have already passed its peak.

We will never achieve AGI, because we keep moving the goalposts.

SOTA models are already capable of outperforming any human on earth in a dizzying array of ways, especially when you consider scale.

Humans also produce nonsensical, useless output. Lots of it.

Yes, LLMs have many limitations that humans easily transcend.

But few if any humans on earth can demonstrate the breadth and depth of competence that a SOTA model possesses.

Relatively few (probably less than half) are casually capable of the level of reasoning that LLMs exhibit.

And, more importantly, as anyone in the field when neural networks were new is aware, AGI never meant human level intelligence until the LLM age. It just meant that a system could generalize one domain from knowledge gained in other domains without supervision or programming.

> It's a hard problem

So people say.

I’m not sidestepping the Hard Problem. I am denying it head on. It’s not a trick or a dodge! It’s a considered stance.

I'm denying that an idea that has historically resisted crisp definition, and that the Stanford Encyclopedia of Philosophy introduces as 'protean', needs to be taken seriously as an essential missing part of AI systems, until someone can explain why.

In my view, the only value the Hard Problem has is to capture a feeling people have about intelligent systems. I contend that this feeling is an artifact of being a social ape, and it entails nothing about AI.