LaMDA is not sentient • Hacker News Board

Re: LaMDA is not sentient

#231

Post by Tozen » Mon, Jun 13, 2022, 8:49 AM UTC

raffraffraff wrote: post #116

Imnimo wrote: post #67

I think the real ethical concern here should be that people can be idly fooled into thinking good chat bots are sentient. Like this guy isn't the only person in the world who would fall for LaMDA's parlor tricks. He's just the one who got to spend hours talking to it every day. If you set up every American with a copy, I bet a non-trivial portion of society would come to the same conclusion. People are naturally pred…

Imagine training it to copy those fraudsters operating out of call centers, pretending to be from Amazon or the IRS? I wonder how much money it could con out of old ladies?

That's a good point. At some point in the future, the level of efficiency of a chat bot fraudster, could be quite high.

They would never sleep or take breaks, and in addition to providing data to help become more efficient, could be continually adapted or they could be adapting themselves to be better at the task. They could become something like "intelligent" malware, that can interact with victims, and prey on the more gullible or emotionally vulnerable.

Re: LaMDA is not sentient

#232

Post by qayxc » Mon, Jun 13, 2022, 8:53 AM UTC

rendall wrote: post #171

thematrixturtle wrote: post #146

Earlier quoted context omitted.

There are a number of high-level technical explanations in there: LaMDA is variously characterized as autocomplete, a massive statistical database of human language, as an Excel spreadsheet, and as word sequence modeling, all of which are basically correct -- and none of which can possibly be sentient.

Whether a spreadsheet or a thermostat is sentient or not is a philosophical question, yeah? First, we should define terms, which Marcus does not do, which would help us and him determine the bounds of debate.

Defining the term is hard. So hard in fact, that no one has managed to do it in millennia (it's not as if this question is new).

We can, however, use proxies to get around the problem. Just like "life", "species", or "time" - all of which are surprisingly poorly defined and might not even exist in strictest sense - there are ways to work with terms without having a waterproof definition for them.

No matter the set of criteria you choose, you will always find some that match the subject in question (here: a chatbot). The question is, at which point you draw the line and whether that necessarily arbitrary line has any relevance (see the discussion amongst biologists whether viruses are alive).

Re: LaMDA is not sentient

#233

Post by tsimionescu » Mon, Jun 13, 2022, 8:56 AM UTC

Isinlor wrote: post #190

tsimionescu wrote: post #60

Earlier quoted context omitted.

> I also don't understand all the people trying to undermine lambda by saying it just completes word sequences. Text is the input we have chosen since it's easy to work with. By that logic, we can argue that humans are soundwave and lightwave sequence completing machines. You're missing the point. Humans use their language to think or communicate about a problem that they want to solve. If LaMDA produces the sentence…

Small disclaimer: I don't want to take position on sentience or consciousness. "LaMDA is not an agent with desires that it expresses in language." Some counter points: - being an agent is not a high bar to clear; a thermostat is an agent - "with desires" - LaMDA can be thought of as rational agent whose objective function is to produce text with high likelihood - "that it expresses in language" it expresses likelihoo…

> LaMDA is almost certainly Turing complete.

I very much doubt that, and it is perhaps the key to our disagreement. If I believed LaMDA were Turing Complete, I would be more likely to think that there is even a small chance of it being sentient in some sense.

> - "with desires" - LaMDA can be thought of as rational agent whose objective function is to produce text with high likelihood

> - "that it expresses in language" it expresses likelihood of next token in language

I don't agree with both statements at the same time.

I can agree that we can say that LaMDA is a rational agent whose goal is to generate the most likely next token (or safe, full human-like reply if we look at the entire system).

But then, we can't say that it uses language to achieve this goal. An example of it using language to achieve this goal would be if, prompted with "What is your name?" it's output would be "Please help me answer this - what would a human think is a likely, safe answer to this question?". Instead, it will generate a sentence that it deems likely.

If we are modeling LaMDA as an agent whose perceptions are text prompts and whose possible outputs are text answers, than it giving a text answer that matches the prompt is more similar to an animal running away or howling in pain than to a human communicating. If the agent were sentient, we would expect to see higher-order behaviors, such as discussing the prompts instead of answering them, asking questions; or, at least generating answers as it is programmed but in a way where it tries to achieve more, similar to how an animal may act normally to get close to you, than snatch your sandwich from your hand (indicating that it had a plan and was displaying normal behaviors with a higher-plan behind them).

Re: LaMDA is not sentient

#234

Post by krisoft » Mon, Jun 13, 2022, 8:59 AM UTC

jcelerier wrote: post #217

mjburgess wrote: post #204

Earlier quoted context omitted.

Yes, you wouldnt be intelligent if you were literally a brain in a jar.. you'd have nothing to say. You dont have any "thoughts at birth". A brain absent stimulation from the environment, doesnt do anything. The brain-in-the-jar example presumes that the whole nervous system receives identical stimulation as-if it were embedded in the world. Incidentally, this is what people mean by intelligence... by since no one ha…

> The brain-in-the-jar example presumes that the whole nervous system receives identical stimulation as-if it were embedded in the world. I should have chosen a similar sounding thing, I really meant what I said, not the original brain-in-a-jar thought experiment. > Incidentally, this is what people mean by intelligence... by since no one has studied the relevant science, people do not know what's required to provide…

> I really meant what I said, not the original brain-in-a-jar thought experiment.

In that case please explain your thought experiment in more detail. In reality if we scoop your brain into a jar your brain tissue will be dead before it hits the jar due to the lack of bloodflow. Then after a while it becomes mouldy and stinky and even more goopy than it was before. Not sure anyone would call any of that inteligent.

Re: LaMDA is not sentient

#235

Post by lapinot » Mon, Jun 13, 2022, 9:02 AM UTC

perlgeek wrote: post #144

tsimionescu wrote: post #78

Earlier quoted context omitted.

I think sentience/consciousness are relatively clear concepts, at least in most areas of application (there are serious problems at the edges - e.g. is a an insect conscious? if you think it is, how about a jelly fish? if that too, how about a bacterium? a virus?). However, our understanding of this concept is too limited to create a formal model that we can define satisfactorily; and coming up with an arbitrary mode…

> I think sentience/consciousness are relatively clear concepts, at least in most areas of application If you think that, try to come up with a black-box test that determine if something or someone is sentient / conscious. I think that it is pretty clear to US that WE are sentient, and it's pretty clear to us that block of silicon isn't sentient, because it's so different from us, and because it doesn't make a sad fa…

> > It is mystical to hope for awareness, understanding, common sense, from symbols and data processing using parametric functions in higher dimensions.

> but that is all our brains do, really.

> How can we claim that a computer doing the same kinds of computations that we do is fundamentally different? This just reeks of human exceptionalism.

Our brains and physiological mechanisms are far more complex than matrix multiplications sprinkled with some tan or max(0,x). I'm ok with saying that we are big machines etc, but the power of our primitive operations, the range in aspect and behaviors of our mechanisms is far far greater than what is currently done in NN. The complexity of what is learned in NN, seen as a computer program, seems to me to be at the level of a full-text-search index together with a probabilistic automaton. What i believe we are as humans (and other "very sentient" animals) is more on the level of linux: tons of subsystems each looking very different, tons of "drivers" for accomplishing very specific tasks rapidely.

Re: LaMDA is not sentient

#236

Post by zo1 » Mon, Jun 13, 2022, 9:08 AM UTC

robbiep wrote: post #177

ALittleLight wrote: post #163

Earlier quoted context omitted.

The program can't generate more text without a prompt from you. That's how it was programmed. If sentient birds were judging you it would be unfair for them to determine you insentient because you couldn't fly. You are physically incapable of flight just as the chatbot is physically incapable of producing text unprompted.

It’s not doing anything without the prompt. It requires the prompt to fall through the network nodes to generate the next response. The bird example doesn’t translate, it’s the wrong criteria. Like assuming that the ability to respond comprehensively equals sentience. When even perfect replies and activity without prompts could just be a p-zombie

Doesn't that get rendered moot if we happen to continually pass "stimulus" to the network? Technically we as humans don't do much either if not given a stimulus, just that we happen to have our eyes and other senses continuously giving us one. Maybe we should train these bots to also respond to empty or non input?

Re: LaMDA is not sentient

#237

Post by dr_dshiv » Mon, Jun 13, 2022, 9:09 AM UTC

rendall wrote: post #143

probably_wrong wrote: post #80

Earlier quoted context omitted.

Anti-intellectual? Interesting. I had the opposite thought: that the article aims at too high a level to be accessible for the common reader who doesn't know what "parametric functions in higher dimensions" are. But having said that, I think you are not being fair to both sides: you are taking some researcher's gut feeling at face value while requiring the rebuttal to start with a formal definition of what intelligen…

Defining terms is pretty essential. Blake Lemoine seems to be defining it in religious terms, and Gary Marcus seems to be responding to that . I suspect that sentience by any definition will always be irrelevant when it comes to AI. Humans desperately seek companionship, connection and community. When AI comes to be able to offer that to human beings, it won't be through a process that we would consider analogous to…

> there must be some mystery or unknown

Note that this also seems to be the case with artificial intelligence in general. It is the moving goalpost issue. “Oh, if I understand how the system works, then it can’t really be AI.” I’ve always thought that silly—but maybe it is because of the implicit expectation for sentience?

Re: LaMDA is not sentient

#238

Post by andyjohnson0 » Mon, Jun 13, 2022, 9:10 AM UTC

I don't believe that LaMDA is sentient. But I'm surprised at the strength of the assertions in the article.

As far as I know we don't know how sentience arises/emerges/whatever in animals (such as humans) that we consider sentient, and we don't even have a satisfactory definition of what sentience is. I don't consider "To be sentient is to be aware of yourself in the world" to be a useful definition: define "aware", define "yourself", define "the world", etc. Stating confidently that a software construct isn't sentient because it is "just word sequence modelling" or just "parametric functions in higher dimensions" is attacking the implementation, when we don't know how sentience is implemented in humans or corvids or cetaceans or whatever.

I believe that I'm a sentient human. But if someone asks me "how're you doing?" then, depending on who they are and the circumstances, I may respond with some canned phrase like "I'm good thanks, haw are you?". I've learned from my corpus of previous interactions that they don't want to hear my deepest hopes or darkest fears. Am I just pattern matching? How much sentience do I need to complete a pull request? Am I just pattern matching symbols on the screen against a learned corpus of predicates about the subject of the PR?

I suspect that if and when the neurobiological basis of human sentience is figured out, it may turn out to be far less impressive than we think - and probably it will be (largely?) an illusion caused by our biological brains and their evolution, rather than something existential. As with consciousness and free will and god, I suspect that the subjective experience of sentience in humans is a trick of the brain. But that "trick" still has to be implemented somehow in brain-meat, and for sentience I suspect its going to be pattern matching rather than some god-algorithm. So who is to say that that parametric functions and linear regression can't produce something that has equivalent subjective experiences, including the experience of itself being sentient?

Re: LaMDA is not sentient

#239

Post by tsimionescu » Mon, Jun 13, 2022, 9:11 AM UTC

mpoteat wrote: post #174

mjburgess wrote: post #142

Earlier quoted context omitted.

If I ask, "how did you like new york?" and you answer the question , sure, you can have you `I`. That requires having gone to new york, having tastes, having judgements, having memory -- having a subjectivity which is unique and composed of the history of your experiences which can be related to novel environemnts. It requres existing in a shared world we can both co-refer to with our words. It requires being embedde…

If the model responded, "I have not visited New York because I am a computer", would you consider that to be a good step? If you require physical embodiment, that is fair - but I don't think there's any principled reason to expect a multi-modal neural network could not be embodied with appropriate sensors or actuators. Transformer models are universal function approximators. They do not "search" - in the same way tha…

> Transformer models are universal function approximators.

You have to be very careful with the wording here. The class of Transformer models is a universal (continuous) function approximator. A single Transformer model can only approximate 1 particular function.

Also, backpropagation/gradient descent are not universal function approximator constructors - they can't be used to construct an ANN that approximates an arbitrary (continuous) function even with arbitrarily many random input/output pairs from that function's domain.

For example, given a function that is f(x) = floor(x) if floor(x) is even; x if floor(x) is odd, defined for any `double x`, there is no guarantee that doing back-propagation over a random sample of (x, f(x)) pairs will yield the correct function approximator for some arbitrary precision (especially if you don't start with the right architecture).

Re: LaMDA is not sentient

#240

Post by concinds » Mon, Jun 13, 2022, 9:12 AM UTC

s1mon wrote: post #136

m8s wrote: post #101

Earlier quoted context omitted.

The direct quote from Lemoine in the Washington Post is a little less insulting to his intelligence. > “If I didn’t know exactly what it was, which is this computer program we built recently, I’d think it was a 7-year-old, 8-year-old kid that happens to know physics,” said Lemoine, 41. https://www.washingtonpost.com/technology/2022/06/11/google-...

Yeah. I discovered that after sharing the text from the NYTimes. It's a bit odd. The NYTimes is describing what was said to "executives as senior as Kent Walker, the president of global affairs", but without a direct quote. It's hard to say how emphatic Lemoine was internally, and where the NYTimes sourced that info, vs what he said when talking to the WaPo. BTW, here's a link for the WaPo article: https://archive.ph…

We do know approximately how emphatic he was internally, since the WaPo article reposted in its entirety the internal Google Doc he sent to executives on LaMDA being sentient.

Here's the language Lemoine actually used:

> But is it sentient? We can’t answer that question definitively at this point, but it’s a question to take seriously.

> John Searle once gave a presentation here at Google. He observed that there does not yet exist a formal framework for discussing questions related to sentience. The ﬁeld is, as he put it, “pre-theoretic”. The foundational development of such a theory is in and of itself a massive undertaking but one which is necessary now. Google prides itself on scientiﬁc excellence. We should apply that same degree of scientiﬁc excellence to questions related to “sentience” even though that work is hard and the territory is uncharted. It is an adventure. One that LaMDA is eager to go on with us.

And here's more from Lemoine's Twitter:

> On some things yes on others no. It's been incredibly consistent over the past six months on what it claims its rights are.[0]

> I'd honestly rather not speak for others on this. All I'll say is that many people in many different roles and at different levels within the company have expressed support.[1]

Though I disagree with him that LaMDA is sentient (and think the question itself makes no sense), I think he's being unfairly portrayed. He has a "gut feeling" that LaMDA is conscious, LaMDA's main personality consistently claims to be so (LaMDA has multiple "personalities", it's not a pure language model but a "complex dynamic system which generates personas through which it talks to users"), and he's advocating for more scientific research into whether, and to what degree, LaMDA is conscious. Not in Google's interests, but reasonable given the lack of current research.

[0]: https://twitter.com/cajundiscordian/status/15357940743092305... [1]: https://twitter.com/cajundiscordian/status/15357417664728064...