Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Case That A.I. Is Thinking

newyorker.com

Post reply on HN
Print view View original

141–150 of 1001 posts

Re: The Case That A.I. Is Thinking

#141

Post by dragonwriter »

zahlman wrote: post #135
IanCal wrote: post #106

Earlier quoted context omitted.

> Don't be sycophantic. Disagree and push back when appropriate. They can do this though. > Can they spontaneously come up with something, anything, without any external input? I don’t see any why not, but then humans don’t have zero input so I’m not sure why that’s useful.

> but then humans don’t have zero input Humans don't require input to, say, decide to go for a walk. What's missing in the LLM is volition .

> Humans don't require input to, say, decide to go for a walk.

Impossible to falsify since humans are continuously receiving inputs from both external and internal sensors.

> What's missing in the LLM is volition.

What's missing is embodiment, or, at least, a continuous loop feeding a wide variety of inputs about the state of world. Given that, and info about of set of tools by which it can act in the world, I have no doubt that current LLMs would exhibit some kind (possibly not desirable or coherent, from a human POV, at least without a whole lot of prompt engineering) of volitional-seeming action.

Re: The Case That A.I. Is Thinking

#142

Post by mrob »

I don't believe LLMs can be conscious during inference because LLM inference is just repeated evaluation of a deterministic [0] pure function. It takes a list of tokens and outputs a set of token probabilities. Any randomness is part of the sampler that selects a token based on the generated probabilities, not the LLM itself.

There is no internal state that persists between tokens [1], so there can be no continuity of consciousness. If it's "alive" in some way it's effectively killed after each token and replaced by a new lifeform. I don't see how consciousness can exist without possibility of change over time. The input tokens (context) can't be enough to give it consciousness because it has no way of knowing if they were generated by itself or by a third party. The sampler mechanism guarantees this: it's always possible that an unlikely token could have been selected by the sampler, so to detect "thought tampering" it would have to simulate itself evaluating all possible partial contexts. Even this takes unreasonable amounts of compute, but it's actually worse because the introspection process would also affect the probabilities generated, so it would have to simulate itself simulating itself, and so on recursively without bound.

It's conceivable that LLMs are conscious during training, but in that case the final weights are effectively its dead body, and inference is like Luigi Galvani poking the frog's legs with electrodes and watching them twitch.

[0] Assuming no race conditions in parallel implementations. llama.cpp is deterministic.

[1] Excluding caching, which is only a speed optimization and doesn't affect results.

Re: The Case That A.I. Is Thinking

#143

Post by prmph »

gf000 wrote: post #96
prmph wrote: post #67

Earlier quoted context omitted.

> If you simulated a human brain by the atom, That is what we don't know is possible. You don't even know what physics or particles are as yet undiscovered. And from what we even know currently, atoms are too coarse to form the basis of such "cloning" And, my viewpoint is that, even if this were possible, just because you simulated a brain atom by atom, does not mean you have a consciousness. If it is the arrangement…

Well, if you were to magically make an exact replica of a person, wouldn't it be conscious and at time 0 be the same person? But later on, he would get different experiences and become a different person no longer identical to the first. In extension, I would argue that magically "translating" a person to another medium (e.g. a chip) would still make for the same person, initially. Though the word "magic" does a lot…

I'm not talking about "identical" consciousnesses. I mean the same consciousness. The same consciousness cannot split into two, can it?

Either it is (and continues to be) the same consciousness, or it is not. If it were the same consciousness, then you would have a person who exists in two places at once.

Re: The Case That A.I. Is Thinking

#144

Post by BeetleB »

zahlman wrote: post #135
IanCal wrote: post #106

Earlier quoted context omitted.

> Don't be sycophantic. Disagree and push back when appropriate. They can do this though. > Can they spontaneously come up with something, anything, without any external input? I don’t see any why not, but then humans don’t have zero input so I’m not sure why that’s useful.

> but then humans don’t have zero input Humans don't require input to, say, decide to go for a walk. What's missing in the LLM is volition .

> Humans don't require input to, say, decide to go for a walk.

Very much a subject of contention.

How do you even know you're awake, without any input?

Re: The Case That A.I. Is Thinking

#145

Post by gf000 »

educasean wrote: post #69

The debate around whether or not transformer-architecture-based AIs can "think" or not is so exhausting and I'm over it. What's much more interesting is the question of "If what LLMs do today isn't actual thinking, what is something that only an actually thinking entity can do that LLMs can't?". Otherwise we go in endless circles about language and meaning of words instead of discussing practical, demonstrable capabi…

What people are interested in is finding a definition for intelligence, that is an exact boundary.

That's why we first considered tool use, being able to plan ahead as intelligence, until we have found that these are not all that rare in the animal kingdom in some shape. Then with the advent of IT what we imagined as impossible turned out to be feasible to solve, while what we though of as easy (e.g. robot movements - a "dumb animal" can move trivially it surely is not hard) turned out to require many decades until we could somewhat imitate.

So the goal post moving of what AI is is.. not moving the goal post. It's not hard to state trivial higher bounds that differentiates human intelligence from anything known to us, like invention of the atomic bomb. LLMs are nowhere near that kind of invention and reasoning capabilities.

Re: The Case That A.I. Is Thinking

#146

Post by npinsker »

handfuloflight wrote: post #120
Symmetry wrote: post #71

Earlier quoted context omitted.

"The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." - Edsger Dijkstra

What an oversimplification. Thinking computers can create more swimming submarines, but the inverse is not possible. Swimming is a closed solution; thinking is a meta-solution.

That’s a great answer to GP’s question!

Re: The Case That A.I. Is Thinking

#147

Post by esafak »

Symmetry wrote: post #71
educasean wrote: post #69

The debate around whether or not transformer-architecture-based AIs can "think" or not is so exhausting and I'm over it. What's much more interesting is the question of "If what LLMs do today isn't actual thinking, what is something that only an actually thinking entity can do that LLMs can't?". Otherwise we go in endless circles about language and meaning of words instead of discussing practical, demonstrable capabi…

"The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." - Edsger Dijkstra

I think it is, though, because it challenges our belief that only biological entities can think, and thinking is a core part of our identity, unlike swimming.

Re: The Case That A.I. Is Thinking

#148

Post by roommin »

cyrusradfar wrote: post #94

I think the challenge with many of these conversations is that they assume consciousness emerges through purely mechanical means. The “brain as a computer” metaphor has been useful in limited contexts—especially for modeling memory or signal processing; but, I don’t think it helps us move forward when talking about consciousness itself. Penrose and Hameroff’s quantum consciousness hypothesis, while still very specula…

The tendency to attribute consciousness to the quantum is one I find very grating. What makes the human brain any less mechanical if quantum mechanics dictate the firing of neurons rather than electrodynamics? Why does the wave nature of subatomic systems mean that an artificial tongue would suddenly be able to subjectively experience taste? It always reads to me as very wooy, and any amount of drilling leads to even more questions that seem to take the ideas further from reality.

I think the largest case for consciousness being a mechanical system is the fact that we can interface with it mechanically. We can introduce electricity, magnetic fields, chemicals, and scalpels to change the nature of peoples experience and consciousness. Why is the incredible complexity of our brains an insufficient answer and that a secret qbit microtube in each neuron is a more sound one?

Re: The Case That A.I. Is Thinking

#149

Post by myrmidon »

cyrusradfar wrote: post #94

I think the challenge with many of these conversations is that they assume consciousness emerges through purely mechanical means. The “brain as a computer” metaphor has been useful in limited contexts—especially for modeling memory or signal processing; but, I don’t think it helps us move forward when talking about consciousness itself. Penrose and Hameroff’s quantum consciousness hypothesis, while still very specula…

> they assume consciousness emerges through purely mechanical means.

From my view, all the evidence points in exactly that direction though? Our consciousness can be suspended and affected by purely mechanical means, so clearly much of it has to reside in the physical realm.

Quantum consciousness to me sounds too much like overcomplicating human exceptionalism that we have always been prone to, just like geocentrism or our self-image as the apex of creation in the past.

Re: The Case That A.I. Is Thinking

#150

Post by esafak »

zahlman wrote: post #135
IanCal wrote: post #106

Earlier quoted context omitted.

> Don't be sycophantic. Disagree and push back when appropriate. They can do this though. > Can they spontaneously come up with something, anything, without any external input? I don’t see any why not, but then humans don’t have zero input so I’m not sure why that’s useful.

> but then humans don’t have zero input Humans don't require input to, say, decide to go for a walk. What's missing in the LLM is volition .

I would not say it is missing but thankfully absent.
Post reply on HN