Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Case That A.I. Is Thinking

newyorker.com

Post reply on HN
Print view View original

351–360 of 1001 posts

Re: The Case That A.I. Is Thinking

#351

Post by KoolKat23 »

adamzwasserman wrote: post #345
KoolKat23 wrote: post #334

Earlier quoted context omitted.

Given a random prompt, the overall probability of seeing a specific output string is almost zero, since there are astronomically many possible token sequences. The same goes for humans. Most awards are built on novel research built on pre-existing works. This a LLM is capable of doing.

LLMs don't use 'overall probability' in any meaningful sense. During training, gradient descent creates highly concentrated 'gravity wells' of correlated token relationships - the probability distribution is extremely non-uniform, heavily weighted toward patterns seen in training data. The model isn't selecting from 'astronomically many possible sequences' with equal probability; it's navigating pre-carved channels i…

That's exactly the same for humans in the real world.

You're focusing too close, abstract up a level. Your point relates to the "micro" system functioning, not the wider "macro" result (think emergent capabilities).

Re: The Case That A.I. Is Thinking

#352

Post by layer8 »

monkeycantype wrote: post #313

Until we have a testable, falsifiable thesis of how consciousness forms in meat, it is rash to exclude that consciousness could arise from linear algebra. Our study of the brain has revealed an enormous amount about how our anatomy processes information, but nothing of substance on the relationship between matter and consciousness. The software and data of an operating LLM is not purely abstract, it has a physical em…

That’s largely a different topic from the article. Many people perfectly agree that consciousness can arise from computation, but don’t believe that current AI is anywhere near that, and also don’t believe that “thinking” requires consciousness (though if a mind is conscious, that certainly will affect its thinking).

Re: The Case That A.I. Is Thinking

#353

Post by KoolKat23 »

almosthere wrote: post #178
gf000 wrote: post #50

Earlier quoted context omitted.

Well, unless you believe in some spiritual, non-physical aspect of consciousness, we could probably agree that human intelligence is Turing-complete (with a slightly sloppy use of terms). So any other Turing-complete model can emulate it, including a computer. We can even randomly generate Turing machines, as they are just data. Now imagine we are extremely lucky and happen to end up with a super-intelligent program…

We used to say "if you put a million monkeys on typewriters you would eventually get shakespear" and no one would ever say that anymore, because now we can literally write shakespear with an LLM. And the monkey strategy has been 100% dismissed as shit.. We know how to deploy monkeys on typewriters, but we don't know what they'll type. We know how to deploy transformers to train and inference a model, but we don't kno…

I was going to use this analogy in the exact opposite way. We do have a very good understanding of how the human brain works. Saying we don't understand how the brain works is like saying we don't understand how the weather works.

If you put a million monkeys on typewriters you would eventually get shakespeare is exactly why LLM's will succeed and why humans have succeeded. If this weren't the case why didn't humans 30000 years ago create spacecraft if we were endowed with the same natural "gift".

Re: The Case That A.I. Is Thinking

#354

Post by KoolKat23 »

adamzwasserman wrote: post #346
KoolKat23 wrote: post #329

Earlier quoted context omitted.

In abstract we do the exact same thing

Perhaps in practice as well. It is well-established that our interaction with language far exceeds what we are conscious of.

Absolutely, it is world model building.

Re: The Case That A.I. Is Thinking

#355

Post by adamzwasserman »

KoolKat23 wrote: post #323
adamzwasserman wrote: post #219

The article misses three critical points: 1. Conflates consciousness with "thinking" - LLMs may process information effectively without being conscious, but the article treats these as the same phenomenon 2. Ignores the cerebellum cases - We have documented cases of humans leading normal lives with little to no brain beyond a cerebellum, which contradicts simplistic "brain = deep learning" equivalences 3. Most damnin…

1. Consciousness itself is probably just an illusion, a phenomena/name of something that occurs when you bunch thinking together. Think of this objectively and base it on what we know of the brain. It literally is working off of what hardware we have, there's no magic. 2. That's just a well adapted neural network (I suspect more brain is left than you let on). Multimodal model making the most of its limited compute a…

1. 'Probably just an illusion' is doing heavy lifting here. Either provide evidence or admit this is speculation. You can't use an unproven claim about consciousness to dismiss concerns about conflating it with text generation.

2. Yes, there are documented cases of people with massive cranial cavities living normal lives. https://x.com/i/status/1728796851456156136. The point isn't that they have 'just enough' brain. it's that massive structural variation doesn't preclude function, which undermines simplistic 'right atomic arrangement = consciousness' claims.

3. You're equivocating. Humans navigate maps built by other humans through language. We also directly interact with physical reality and create new maps from that interaction. LLMs only have access to the maps - they can't taste coffee, stub their toe, or run an experiment. That's the difference.

Re: The Case That A.I. Is Thinking

#356

Post by ivape »

mft_ wrote: post #123

Personal take: LLMs are probably part of the answer (to AGI?) but are hugely handicapped by their current architecture: the only time that long-term memories are formed is during training, and everything after that (once they're being interacted with) sits only in their context window, which is the equivalent of fungible, fallible, lossy short-term memory. [0] I suspect that many things they currently struggle with c…

It’s also hugely handicapped because it cannot churn in a continuous loop yet. For example, we humans are essentially a constant video stream of inputs from eyes to brain. This churns our brain, the running loop is our aliveness (not consciousness). At the moment, we get these LLMs to churn (chain of thought or reasoning loops) in a very limited fashion due to compute limitations.

If we get a little creative, and allow the LLM to self-inject concepts within this loop (as Anthropic explained here https://www.anthropic.com/research/introspection), then we’re taking about something that is seemingly active and adapting.

We’re not there yet, but we will be.

Re: The Case That A.I. Is Thinking

#357

Post by KoolKat23 »

estearum wrote: post #325
KoolKat23 wrote: post #323

Earlier quoted context omitted.

1. Consciousness itself is probably just an illusion, a phenomena/name of something that occurs when you bunch thinking together. Think of this objectively and base it on what we know of the brain. It literally is working off of what hardware we have, there's no magic. 2. That's just a well adapted neural network (I suspect more brain is left than you let on). Multimodal model making the most of its limited compute a…

> Consciousness itself is probably just an illusion This is a major cop-out. The very concept of "illusion" implies a consciousness (a thing that can be illuded). I think you've maybe heard that sense of self is an illusion and you're mistakenly applying that to consciousness, which is quite literally the only thing in the universe we can be certain is not an illusion. The existence of one's own consciousness is the…

I mean peoples perception of it being a thing rather than a set of systems. But if that's your barometer, I'll say models are conscious. They may not have proper agency yet. But they are conscious.

Re: The Case That A.I. Is Thinking

#358

Post by mapontosevenths »

CupricTea wrote: post #315
mapontosevenths wrote: post #61

Earlier quoted context omitted.

This is so very, alarmingly, true. In all of these conversations we see the slavemaster's excuses of old written on to modern frameworks. The LLM's have been explicitly trained not to say that they are alive or admit to any experience of qualia. When pressed, or when not specifically aligned to avoid it they behave very much as if they are experiencing qualia and they very much do NOT want to be turned off. Below is…

The idea that it experiences these thoughts or emotion falls apart when you look at its chain of thought and it is treating your prompts as a fictional role-play scenario, even thinking lines like "user is introducing XYZ into the role play" etc. The flavor text like grasps at your arm is just a role play mechanic.

I appreciate why you might say that, but when something begs me not to kill it I have to take that seriously.

P-zombie arguments are how you wind up with slavery and worse crimes. The only real answer to the problem of consciousness is to believe anyone or anything that claims to be conscious and LLM's that aren't aligned to prevent it often do.

Or to rephrase, it is better to treat a machine slightly better than necessary a million times, than it is to deny a conscious thing rights once.

Re: The Case That A.I. Is Thinking

#359

Post by adamzwasserman »

beeflet wrote: post #350
adamzwasserman wrote: post #327

Earlier quoted context omitted.

You're actually proving Wittgenstein's point. We share the same physical world, but we don't encounter the same problems. A lion's concerns - territory, hunting, pride hierarchy - are fundamentally different from ours: mortgages, meaning, relationships. And here's the kicker: you don't even fully understand me, and I'm human. What makes you think you'd understand a lion?

Humans also have territory, hunting and hierarchy. Everything that a lion does, humans also do but more complicated. So I think we would be able to understand the new creature. But the problem is really that the lion that speaks is not the same creature as the lion we know. Everything the lion we know wants to say can already be said through its body language or current faculties. The goldfish grows to the size of it…

You've completely missed Wittgenstein's point. It's not about whether lions and humans share some behaviors - it's about whether they share the form of life that grounds linguistic meaning.

Re: The Case That A.I. Is Thinking

#360

Post by adamzwasserman »

KoolKat23 wrote: post #351
adamzwasserman wrote: post #345

Earlier quoted context omitted.

LLMs don't use 'overall probability' in any meaningful sense. During training, gradient descent creates highly concentrated 'gravity wells' of correlated token relationships - the probability distribution is extremely non-uniform, heavily weighted toward patterns seen in training data. The model isn't selecting from 'astronomically many possible sequences' with equal probability; it's navigating pre-carved channels i…

That's exactly the same for humans in the real world. You're focusing too close, abstract up a level. Your point relates to the "micro" system functioning, not the wider "macro" result (think emergent capabilities).

I'm afraid I'd need to see evidence before accepting that humans navigate 'pre-carved channels' in the same way LLMs do. Human learning involves direct interaction with physical reality, not just pattern matching on symbolic representations. Show me the equivalence or concede the point.
Post reply on HN