Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Case That A.I. Is Thinking

newyorker.com

Post reply on HN
Print view View original

391–400 of 1001 posts

Re: The Case That A.I. Is Thinking

#391

Post by omnicognate »

tim333 wrote: post #388
omnicognate wrote: post #151

Earlier quoted context omitted.

> "If what LLMs do today isn't actual thinking, what is something that only an actually thinking entity can do that LLMs can't?" Independent frontier maths research, i.e. coming up with and proving (preferably numerous) significant new theorems without human guidance. I say that not because I think the task is special among human behaviours. I think the mental faculties that mathematicians use to do such research are…

That's quite a high bar for thinking like humans which rules out 99.99% of humans.

I have never claimed that only people/machines that can do frontier maths research can be intelligent. (Though someone always responds as if I did.)

I said that a machine doing frontier maths research would be sufficient evidence to convince me that it is intelligent. My prior is very strongly that LLM's do not think like humans so I require compelling evidence to conclude that they do. I defined one such possible piece of evidence, and didn't exclude the possibility of others.

If I were to encounter such evidence and be persuaded, I would have to also consider it likely that LLMs employ their intelligence when solving IMO questions and generating code. However, those tasks alone are not sufficient to persuade me of their intelligence because I think there are ways of performing those tasks without human-like insight (by interpolating/recombining from vast training data).

As I said elsewhere in this thread:

> The special thing about novel mathematical thinking is that it is verifiable, requires genuine insight and is a text generation task, not that you have to be able to do it to be considered intelligent.

Re: The Case That A.I. Is Thinking

#392

Post by paxys »

BeetleB wrote: post #140
paxys wrote: post #100

Earlier quoted context omitted.

Don't be sycophantic. Disagree and push back when appropriate. Come up with original thought and original ideas. Have long term goals that aren't programmed by an external source. Do something unprompted. The last one IMO is more complex than the rest, because LLMs are fundamentally autocomplete machines. But what happens if you don't give them any prompt? Can they spontaneously come up with something, anything, with…

> Disagree and push back The other day an LLM gave me a script that had undeclared identifiers (it hallucinated a constant from an import). When I informed it, it said "You must have copy/pasted incorrectly." When I pushed back, it said "Now you trust me: The script is perfectly correct. You should look into whether there is a problem with the installation/config on your computer."

That's the flip side of the same symptom. One model is instructed to agree with the user no matter what, and the other is instructed to stick to its guns no matter what. Neither of them is actually thinking.

Re: The Case That A.I. Is Thinking

#393

Post by senordevnyc »

b00ty4breakfast wrote: post #276

all this "AI IS THINKING/CONSCIOUS/WHATEVER" but nobody seems worried of that implication that, if that is even remotely true, we are creating a new slave market. This either implies that these people don't actually believes any of this boostering rhetoric and are just cynically trying to cash in or that the technical milieu is in a profoundly disturbing place ethically. To be clear, I don't believe that current AI t…

As I recall a team at Anthropic is exploring this very question, and was soundly mocked here on HN for it.

Re: The Case That A.I. Is Thinking

#394

Post by hackinthebochs »

almosthere wrote: post #372
hackinthebochs wrote: post #212

Earlier quoted context omitted.

This is a bad take. We didn't write the model, we wrote an algorithm that searches the space of models that conform to some high level constraints as specified by the stacked transformer architecture. But stacked transformers are a very general computational paradigm. The training aspect converges the parameters to a specific model that well reproduces the training data. But the computational circuits the model picks…

However there is another big problem, this would require a blob of data in a file to be labelled as "alive" even if it's on a disk in a garbage dump with no cpu or gpu anywhere near it. The inference software that would normally read from that file is also not alive, as it's literally very concise code that we wrote to traverse through that file. So if the disk isn't alive, the file on it isn't alive, the inference s…

This is an overly reductive view of a fully trained LLM. You have identified the pieces, but you miss the whole. The inference code is like a circuit builder, it represents the high level matmuls and the potential paths for dataflow. The data blob as the fully converged model configures this circuit builder in the sense of specifying the exact pathways information flows through the system. But this isn't some inert formalism, this is an active, potent causal structure realized by the base computational substrate that is influencing and being influenced by the world. If anything is conscious here, it would be this structure. If the computational theory of mind is true, then there are some specific information dynamics that realize consciousness. Whether or not LLM training finds these structures is an open question.

Re: The Case That A.I. Is Thinking

#395

Post by Retric »

adamzwasserman wrote: post #288
Retric wrote: post #280

Earlier quoted context omitted.

This is all really arbitrary metrics across such wildly different fields. IMO LLMs are where computer vision was 20+ years ago in terms of real world accuracy. Other people feel LLMs offer far more value to the economy etc.

I understand the temptation to compare LLMs and computer vision, but I think it’s misleading to equate generative AI with feature-identification or descriptive AI systems like those in early computer vision. LLMs, which focus on generating human-like text and reasoning across diverse contexts, operate in a fundamentally different domain than descriptive AI, which primarily extracts patterns or features from data, lik…

I don’t think it’s an oversimplification as accuracy is what constrains LLMs across so many domains. If you’re a wealthy person asking ChatGPT to write a prenup or other contract to use would be an act of stupidity unless you vetted it with an actual lawyer. My most desired use case is closer, but LLMs are still more than an order of magnitude below what I am willing to tolerate.

IMO that’s what maturity means in AI systems. Self driving cars aren’t limited by the underlying mechanical complexity, it’s all about the long quest for a system to make reasonably correct decisions hundreds of times a second for years across widely varying regions and weather conditions. Individual cruse missiles on the other hand only needed to operate across a single short and pre-mapped flight in specific conditions, therefore they used visual navigation decades earlier.

Re: The Case That A.I. Is Thinking

#396

Post by anyonecancode »

kenjackson wrote: post #246
bitwize wrote: post #216

Earlier quoted context omitted.

I'm also reminded of the bit from Neuromancer where Case removes and then reinserts the Dixie Flatline "ROM construct" cartridge, resetting Dixie to the moment just before his death and causing him to forget their previous (albeit brief) conversation. Dixie can't meaningfully grow as a person. All that he ever will be is burned onto that cart; anything he learns since then is stored in temporary memory. Perhaps this…

"Dixie can't meaningfully grow as a person. All that he ever will be is burned onto that cart;" It's not that Dixie can't meaningful grow -- really the issue is that Dixie can be reset. If Dixie's cart simply degraded after 90 years, and you couldn't reset it, but everything else was the same -- would you then say Dixie could grow as a person? As humans we basically have a 90 year cart that once it no longer works, w…

I strongly suspect the answer is yes -- or more broadly, what makes us conscious. And yes, this implies consciousness is something all life has, to some degree.

I'm not going to pretend to have a good definition of what "consciousness" is, but directionally, I think having goals -- no, that's too weak -- having _desires_, is an important part of it. And I'm not sure it's possible to have desires if one cannot die.

Something like an LLM can't actually die. Shut down all the machines its code runs on, then turn them back on, and it's in the same state it was before. So it's not the "hardware" that an LLM lives in. Is it the code itself? Copy it to another set of machines and it's the same program. Code + data? Maybe we run into storage issues, but in theory same thing -- transfer the code and date somemplace else and its the same program. You can't actually "kill" a computer program. So there's no inherent "mortality" to it that where any kinds of "desire" would emerge from.

Re: The Case That A.I. Is Thinking

#397

Post by adamzwasserman »

goatlover wrote: post #387
andoando wrote: post #303

Earlier quoted context omitted.

>I think that Wittgenstein had it right when he said: "If a lion could speak, we could not understand him." Why would we not? We live in the same physical world and encounter the same problems.

We haven't been able to decode what whales and dolphins are communicating. Are they using language? A problem SETI faces is whether we would be able to decode an alien signal. They may be too different in their biology, culture and technology. The book & movie Contact propose that math is a universal language. This assumes they're motivated to use the same basic mathematical structures we do. Maybe they don't care ab…

The whole "math is a universal" language is particularly laughable to me considering it is a formal system and the universe is observably irregular.

As I am wont to say: regularity is only ever achieved at the price of generality.

Re: The Case That A.I. Is Thinking

#398

Post by adamzwasserman »

KoolKat23 wrote: post #362
adamzwasserman wrote: post #360

Earlier quoted context omitted.

I'm afraid I'd need to see evidence before accepting that humans navigate 'pre-carved channels' in the same way LLMs do. Human learning involves direct interaction with physical reality, not just pattern matching on symbolic representations. Show me the equivalence or concede the point.

Language and math are a world model of physical reality. You could not read a book and make sense of it if this were not true. An apple falls to the ground because of? gravity. In real life this is the answer, I'm very sure the pre-carved channel will also lead to gravity.

You're proving my point. You know the word 'gravity' appears in texts about falling apples. An LLM knows that too. But neither you nor the LLM discovered gravity by observing reality and creating new models. You both inherited a pre-existing linguistic map. That's my entire argument about why LLMs can't do Nobel Prize-level work.

Re: The Case That A.I. Is Thinking

#399

Post by adamzwasserman »

KoolKat23 wrote: post #376
adamzwasserman wrote: post #355

Earlier quoted context omitted.

1. 'Probably just an illusion' is doing heavy lifting here. Either provide evidence or admit this is speculation. You can't use an unproven claim about consciousness to dismiss concerns about conflating it with text generation. 2. Yes, there are documented cases of people with massive cranial cavities living normal lives. https://x.com/i/status/1728796851456156136 . The point isn't that they have 'just enough' brain.…

1. What's your definition of consciousness, let's start there. 2. Absolutely, it's a spectrum. Insects have function. 3. "Humans navigate maps built by other humans through language." You said it yourself. They use this exact same data, so why won't they know it if they used it. Humans are their bodies in the physical world.

1. I don't need to define consciousness to point out that you're using an unproven claim ('consciousness is probably an illusion') as the foundation of your argument. That's circular reasoning.

2. 'It's a spectrum' doesn't address the point. You claimed LLMs approximate brain function because they have similar architecture. Massive structural variation in biological brains producing similar function undermines that claim.

3. You're still missing it. Humans use language to describe discoveries made through physical interaction. LLMs can only recombine those descriptions. They can't discover that a description is wrong by stubbing their toe or running an experiment. Language is downstream of physical discovery, not a substitute for it

Re: The Case That A.I. Is Thinking

#400

Post by johnnienaked »

tkz1312 wrote: post #274

Having seen LLMs so many times produce coherent, sensible and valid chains of reasoning to diagnose issues and bugs in software I work on, I am at this point in absolutely no doubt that they are thinking. Consciousness or self awareness is of course a different question, and ones whose answer seems less clear right now. Knee jerk dismissing the evidence in front of your eyes because you find it unbelievable that we c…

If you understand how they operate and you are reasonable and unbiased there is no way you could consider it thinking
Post reply on HN