Artificial intelligence is not conscious – Ted Chiang

Ok, deploy a local model on a lightweight edge compute device and strap it to a chassis with wheels, and attach a cheap webcam

> Then I’d want to see an embodied agent that could navigate its environment in order to survive as well as, say, a lizard can

Give the robot appendages that enable it to plug itself into a standard wall outlet, guided by a vision model plugged into its webcam. As long as it can feed itself, it can survive long enough.

> Next I would want to see an embodied agent with the same capacity to deal with novel situations as a mouse.

I think if you fed frames from the webcam into a local VLM every 5s you’d be able to assess a situation and respond with simple actions (turn, advance, retreat).

> After that I’d want to see agents whose social dynamics are as complex as those of wolves, and then agents with the tool-making abilities of chimpanzees.

Social dynamics could be implemented in many ways, maybe by transmitting tokens over RF? Idk. Then you have a scanner that picks them up, feeds them into some LLM frontend and decides whether to add them to a global context file that guides the VLM action-taker. A new action could be to broadcast a token message. Tool-making would have to be code-based. Physical tools are hard. Still unsolved.

> At that point I would want to see people successfully teaching such embodied agents how to communicate their desires

This part is relatively straightforward except for the “via nonlinguistic modality”.

Anyway. These are all engineering problems. Personally I would demand to see the AI reproduce its body under its own power and volition. That’s a pretty neat trick we’ve got going for us.

Telling models to "think hard" or "go step by step" has at times had an impact on the quality of the output. To deny that is silly. But that is treating it like it's conscious, and to deny that "consciousness" even if correct, does nothing but place an unnecessary burden on the person interacting with it.

I understand that LLMs are "just next word machines" but to constantly maintain that concept in my head while I'm typing "act as a financial expert and think carefully" is a waste of my mental energy.

Similar to: "Birds fly, my spinning helical device flies, therefore we've started to replicate how birds fly."

> without having to build elements that one expect on a conscious being

One of the elements I expect in a conscious being is that you can't rewrite it by changing the introductory paragraph.

When it comes to LLMs, almost every "mind" we humans perceive is a fictional character in an LLM-generated story-document, one we are either reading or which is being "acted" at us by regular code. Our own instinct for pareidolia and simulating/inferring other minds is very strong, which means we should require really good evidence/logic to counter our instincts.

Even if one believes the LLM has a single "real mind" as an author of every document... what evidence do we have that it is conscious or "self-inserting" itself as one of the characters in the document?

You're assuming that because Claude produces text that appears to express these qualities, Claude must have them. I don't think that's a good assumption.

Even many humans produce text that has the same appearance, but don't actually have those qualities--which becomes clear when you look at what they do, not what they say. So the assumption isn't even a valid one for humans. Talk is cheap.

On top of that, Claude doesn't even have the same kinds of connections to the outside world that humans do. All Claude has is text. So if you can't even trust humans to back up their words with actions, you should be much, much less trusting of Claude. Talk is a lot cheaper for Claude than it is for a human.

This is where the other claim is being made. That the structure of the model is fundamentally incapable of the operation, so even if you stipulated that the way you provide data is sufficient for intelligence then it still wouldn't work.

The universal approximation theorem addresses this point. In that, with an identity attention mechanism, a LLM is just a multi layer perceptron. The attention mechanism is effectively a way to get one of the benefits of a much larger fully connected layer without the massive cost.

A LLM can do what a MLP can do. A large enough MLP can do any function to arbitrary precision.

That makes the claim that an LLM could not do a task the same as saying no function can do that task.

Some are ok with this, if you invoke some supernatual aspect to intelligence then the inability to describe it with a function is quite reasonable,

If you want to stay in the world of reality, you have a much harder task, people like to point at quantum (Penrose) but it's hard to say what it is you are pointing at.

I think the very act of proving that something is or is not intelligent, would render it functional by nature of it having a proof, (or disprove Gödel's incompleteness (a tough ask))

Are there any proofs that cannot be expressed as a function? A kind of Gödel locator, where you can prove something that you can identify is true but there is no formula to express it. I'm not entirely sure what that would even mean,