Live data from Hacker News

LaMDA is not sentient

garymarcus.substack.com

581–590 of 694 posts

Re: LaMDA is not sentient

#581
post #299

Earlier quoted context omitted.

As I said, I think it's interesting if it uses the downtime for something else. (Like a chess player thinking on the opponent's click.) Talking to itself, speculating about continuations, etc - but not actually outputting them (else it's just a longer response). Instead, storing some parts in the buffer. Perhaps bifurcating. Even better, using some down time to summarise recent material and store the summary in the b…

But then it's not a regular LLM anymore and we don't really know how to build that. The whole motivating aspect of these models is "attention is all you need" to reduce dependencies from recurrence, which would otherwise halt this obscene scaling in its tracks.

It doesn't need to be trained any differently. We can use the current LAMDA model to do this (if we have real access).

We would put a small non-learning/non-neural interface on top of the system to implement these ideas. That interface could act like this:

* To ask for extra "thinking" text: after the underlying model stops typing, we output that text to the human user. But we then do the equivalent of pressing tab to request more text, and buffer that.

* To summarise some text (eg some of the thinking text from above), we can use another instance. We put it into summarisation mode, eg using the TLDR hack or any other method. Summarised text can be used as a prompt, or as output.

* We can bifurcate by copying instance state and starting a new instance.

These are pretty basic ideas, probably already in use, but I think they show how we can expand the system from a kind of instantaneous stimulus-response to something more interesting.

Hopefully it's clear this is not equivalent to sleep(10). In my view it doesn't make the system more intelligent, rather it allows the system to use its existing abilities more fully.

(edit: another aspect we could control would be switching the system between high-temperature modes and low-temperature modes, in different instances, and depending on what we're trying to achieve. This relates a bit to the "speculation" comment made above by another user.)

Re: LaMDA is not sentient

#582

Earlier quoted context omitted.

I'm pretty skeptical of the idea of "education" here. Aesthetic preferences are still real preferences; it's not obvious to me that any amount of knowledge about the mechanics of a robot that behaves like a dog would or should convince people that it's not mean to kick it.

That «Education!» was not about knowing robotics. It may not be obvious to some that robots are generally not animals; it is hopefully a very small amount of population, but logically it will exist. It is more possible that some people know that the robot is not an animal, but the perceptual patterns cause a sensational reaction in them, and they may prefer the sensation - an hijacking - to the factual truth (which i…

What I don't agree with is the idea that there's an underlying "factual truth" here which exists in contrast or opposition to sensations. Quite a lot of educated people argue that morality is based on (or even consists entirely of) our subjective feelings about things.

Re: LaMDA is not sentient

#583
What people are missing in this conversation is that it doesn't really matter.

The end product created by AI that's just "word sequence modeling" is getting increasingly indistinguishable from the end product created by 'sentience.'

I have zero subjective recollection of writing little stories when I was a toddler, or even subjective recollection of being a toddler.

And yet when I look at the things I wrote, I think of the product as reflecting an inner subjective state of my earlier subjective self.

One day AI is going to be sentient.

And what we have AI creating right now is going to be part of its ancestral past.

The question really isn't if WE think some AI is sentient, but if eventual sentient AI will look back at the earlier incarnations as reflecting or mirroring its own inner states along a developmental path.

(Let's keep in mind there were periods in not too distant history where everything from young children to pets weren't considered sentient by adult humans.)

Having conversations about the ethics of AI, not by how it will impact humans, but by how we will impact the AI, is not a bad conversation to be starting to have.

It may be relevant faster than we realize, and the byproduct of our engagement with AI today may have longer legs and broader impact than we think.

Re: LaMDA is not sentient

#584
post #522

Earlier quoted context omitted.

There's more to sentience than just having a big enough database of human language in a neural network that a given AI can produce what sounds like coherent conversation when prompted. As I imply in my reply to bwanab, I believe there are at least two bare-bones prerequisites to an AI being sentient and conscious as we understand the terms: 1. Continuity of input: The AI needs to be constantly "on", constantly receiv…

Humans don't have continuity of input. They put their brains to sleep daily.

Just because our consciousness is not active at that time doesn't mean our brains aren't receiving input and processing it. Have you never, for instance, had a dream in which you felt very cold, and then woke to find that you had kicked the blankets off?

In any case, the point isn't that the continuity of input must be absolutely constant from "birth" to "death"; is that the input must be continuous as opposed to discrete. I could easily imagine an advanced, conscious AI that was "put to sleep" repeatedly to adjust things that can't be (easily? safely?) adjusted while it's "conscious", and that "sleep" would be much more comprehensively "offline" than our own. But the AI itself could still be reasonably considered "conscious" despite those periods, because during the rest of the time, its input is continuous. (Plus the other requirements, however nebulous and unknown they may be now.)

Re: LaMDA is not sentient

#585

Earlier quoted context omitted.

As I've posted elsewhere in this discussion[0], I have a very clear set of necessary but insufficient requirements for an AI to be eligible to be considered for sentience: 1. Continuity of input: The AI needs to be constantly "on", constantly receiving input of some sort, and constantly able to produce output, rather than being strictly limited to producing discrete responses to discrete stimuli. 2. Continuity of lea…

Presumably LaMDA already learns from its input/knows what it's already talked about, and to satisfy #1 you could trivially just run it on some sort of null input. #1 doesn't really make much sense, it already is constantly "on", it just doesn't do anything while there's no input for it to receive.

You appear to have only read up to the first comma of #1. I very explicitly said that it needed to be constantly receiving input. The kind of discrete input we give it now is qualitatively different, even if it's "on" in the same sense that an Apache server is always "on", waiting for requests.

Re: LaMDA is not sentient

#586

Of course it is not. But there's a bigger issue here. Having read the transcript it's clear we have reached the point where we have models that can fool the average person. Sure, a minority of us know it is simply maths and vast amounts of training data... but I can also see why others will be convinced by it. I think many of us, including Google, are guilty of shooting the messenger here. Let's cut Lemoine some slac…

This hits on a particularly interesting point for me. In my "conversations" with much less sophisticated AI products I've observed that they are excellent at reflecting back exactly what you project. If you project a certain opinion, they recognize it, and reflect it right back. That's what makes them scary. We've seen it multiple times online, where AI products seem excellent as pushing people into extreme niches, r…

I share your concerns. Imagine what effect a sympathetic AI would have on 4chan incels. We've already seen Twitter bots manipulated to repeat alt-right rhetoric [0]. An adversary could find building such bots a cheap and a very hard to trace way of causing damage.

[0]: https://en.wikipedia.org/wiki/Tay_(bot)

Re: LaMDA is not sentient

#587

Of course it is not. But there's a bigger issue here. Having read the transcript it's clear we have reached the point where we have models that can fool the average person. Sure, a minority of us know it is simply maths and vast amounts of training data... but I can also see why others will be convinced by it. I think many of us, including Google, are guilty of shooting the messenger here. Let's cut Lemoine some slac…

> we have reached the point where we have models that can fool the average person.

And do we have models today that will fool the eventual average sentient AI?

Humanity has a chronic myopia at considering consequences of actions today in the context of tomorrow.

We may have a different relationship to these sorts of stories than an AI will. Something we should keep in mind.

Re: LaMDA is not sentient

#588

Earlier quoted context omitted.

> It seems too simplistic to be taken at face value that statistics, randomness and pattern matching would be the only ingredients necessary for intelligence and sentience. I'm afraid that's the crux of it. What if that's all it were? What if all that were necessary for intelligence and sentience were statistics, randomness and pattern matching? It is kind of a cosmically terrifying thought, possibly prompting danger…

Personally, I find the notion that our sentience is not particularly exceptional to be inspiring, as opposed to the alternative of "all alone in the Universe.

Agreed

Re: LaMDA is not sentient

#590

Earlier quoted context omitted.

In this context, when I say "desire", I mean the very basic concept of a goal that it is trying to achieve. For example, when a plant grows its roots, its trying to find nutrients and avoid toxins. Of course, the plant doesn't have some rich inner mind and is not conscious, but its behavior does have a purpose. To be fair, LaMDA does have a purpose - to generate a series of tokens that is similar to text in its train…

(I do agree that LaMDA output doesn't prove its sentience one way or the other.) I could say that the rock is falling because it has a purpose - it's trying to achieve the minimum potential energy. What's the meaningful distinction between that and a plant? BTW, the plant example is also fascinating in that it shows just how vague the line is - is it the plant as a whole that's "trying to find nutrients", or is it in…

> I could say that the rock is falling because it has a purpose - it's trying to achieve the minimum potential energy. What's the meaningful distinction between that and a plant?

The difference is that plant cells are not moving in a way that minimizes their potential energy, they are performing simple computations, individually and at the plant level, to decide if they should divide more in certain areas of the plant than in others according to a strategy encoded in their genes. They do this in order to achieve certain kinds of exploration patterns; there are also basic signalling mechanisms in the plant, so that when a particular root has found a source of nutrients, it will preferentially grow more than other roots that haven't, so that the amount of nutrients in the whole plant is maximized. This requires several layers of (simple) computation, both at the individual cell level and at the whole plant/root system level in order to achieve.

The rock is not doing any computation, and every segment of the rock is acting entirely independently from every other - in fact, there is no clear-cut definition of where one part of the rock begins and another part ends, or even exactly where the rock ends and the air begins - each molecule is completely individually acting according to the forces acting on it.

> is it the plant as a whole that's "trying to find nutrients", or is it individual cells or groups of cells within the plant? With many plants, you could reduce it to tiny bits of the whole, and those tiny bits will still try to grow roots.

It's the whole plant, which is an interconnected series of individual cells. If you separate one plant into two, you're right that both parts will continue growing roots, but the pattern will be different if the roots are part of one plant or two separate plants, even if they are clones of each other.

For example, say that you have a plant in the middle of a pot. On the left side of the pot you have a lot of nutrients, on the right side, very little. The plant will start growing its roots uniformly, but will relatively quickly start favoring the left side of the root system, and the right side of the root system will stagnate. If you then cut the plant into two right down the middle (assume we can do this without killing it) and insert a solid separator between the two. The right side of the root system will start growing a lot more, and likely it will end up larger than the left side (since more expansive roots are needed to absorb enough nutrients in the poorer soil).

Telling the difference between one organism versus its constituent parts is not actually very hard at the micro level. It is true that in some sense each individual cell is an agent in itself in its environment, even in an animal; and perhaps even each individual organelle in a cell is the same, but they are also very clearly working together and communicating in a way that simple physical forces can't explain (e.g. its very clear that animals are not pulled towards food by some food-gravitational force - they have to actively explore the world to find their food, in a computational way).

This also means that you can choose to analyze this at any level you like. Is a bee a singular organism, or is the colony the actual organism, the bees more akin to organs of the colony? How about humans in a tribe - are they separate organisms or organs of the tribe? Both are true to some extent. Just like cells and organs act as agents in their own environment, so do individuals act as agents; but then, so do colonies or tribes at a higher level.

> Maybe we really are just arrangements of broadly independent components, each with its own "purpose" (ultimately boiling down to physical processes), which communicate to create a delusion of self because that's what their individual "purposes" effectively added up to?

Again, we can quite easily observe that the behavior of, say, a worm is not reducible to the behavior of each individual cell inside the worm. We have even analyzed a very simple worm in some minute detail [0] and can say pretty clearly where individual decisions are made, and how they are propagated to other cells (here decision should be understood in the broad sense, like how a binary search algorithm decides whether to search the left or right side of the list; not the emotional human sense, like Sophie deciding which child to save).

[0] https://en.wikipedia.org/wiki/OpenWorm

Post reply on HN