Why AI is still dumb and not scary at all (pt. 1)

A more honest blog post would read something like this: Big Ai models are interesting and useful. And there’s the potential for danger. We should be cautious and make sure we have some bright scholars studying what might happen if various scenarios play out. Just as we would want to be in response to any similarly massive advancement in technology.

What do you think of the latest thinking models, and what is your test of thinking?

Tejo writes:

> At the heart of it, all AI is just a pattern recognition engine. They’re simply algorithms designed to maximize or minimize an objective function. There is no real thinking involved. It’s just math. If you think somewhere in all that probability theory and linear algebra, there is a soul, I think you should reassess how you define sentience.”

We have two words we need to define: “sentience” and “thinking”.

Sentience can be defined as a basic function of most living systems (viruses excluded). You may disagree but to me this word connotes “actively responsive to perturbations”. This is part of the auopoietic definition of life introduced by Maturana and Valera in the 1970s and 1980s. If you buy this definition then “sentience” is a low bar and not what we mean by AI, let alone by AGI.

The word “thinking” implies more than just consciousness. Many of us accept that cats, dogs, pigs, chimps, crows, parrots, and whales are conscious. Perhaps not an ant or a spider, but yes, probably rats since rats play and smile and clearly plan. But few of us would be likely to claim that a rat thinks in a way that we think. Sure, a rat analyzes its environment and comes to decisions about best actions, but that does not require what many of us mean by thinking.

We usually think of thinking as a process of reflection/recursion/self-evaluation of alternative scenarios. The key word here is recursion. and that gets us to “self-consciousness”—a word that embeds recursion.

I operationally define thinking as a process that involves recursive review and reflection/evaluation of alternatives with the final selection of an “action”—either a physical action or an action that is a “thought conclusion”.

What AI/LMM transformers do not have is deep enough recursion or access to memory of preceding states. They cannot remember and they do not have a meta-supervisor that lets them modulate their own attentional state. So I would agree that they are still incapable of what most if us call thinking.

But how hard is adding memory access to any number of current generation AI systems?

How hard us it to add recursion and chain-of-thought?

How hard will it be to enable an AI system to attend to its own attention?

In my optimistic view these are all close to solved problems.

In my opinion the only thing separating us from true AGI are a few crucial innovations in AI architecture—-mainly self-control of attention—-and under 100k lines of code.

Please argue! (I’ve had many arguments with Claude on this topic).

There are already active discussions about AI being used in government for “efficiency” reasons.

There are better ways to argue the same position but it's probably indefensible.

Neural networks are fundamentally approximators, both when they are approximating long-range relations between concepts (like LLMs do) or when they denoise some noise for images. They approximate thought and intelligence, because we have coded it in our writings. Our writings are a complete fingerprint of the thought process, because we are able to pick up any thought process from a book, without ever seeing or otherwise having contact with the author. Therefore there is an increasingly more high-fidelity "real thought" in there.

> that in and of itself is a monumental achievement but there is no real thought involved.

Pretty much anything that requires our current level of thought is therefore reachable with ANNs now that we know how to train them in depth and width.

The real question is whether this is enough. We want ASI, but our texts only have AGI, and everything that comes from biology (including intelligence) is logarithmic in scale. There is zero evidence that language models will ever learn to create abstract entities better than any collection of humans do. AI companies are advertising armies of PhD students, but we already have millions of PhD students , yet our most pressing problems have not made a lot of progress for decades. That's what should be worrying to us, not the fact that we will all lose our jobs.