Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Case That A.I. Is Thinking

newyorker.com

Post reply on HN
Print view View original

891–900 of 1001 posts

Re: The Case That A.I. Is Thinking

#891

Post by ACCount37 »

ViewTrick1002 wrote: post #807
jamincan wrote: post #750

Earlier quoted context omitted.

For what it's worth, when I ask ChatGPT 5, it gets the correct answer every time. The response varies, but the answer is always three.

Now try a different language. My take is hard RL tuning to fix these "gotcha:s" since the underlying model can't do it on its own. OpenAI is working on ChatGPT the application and ecosystem. They have transitioned from model building to software engineering with RL tuning and integration of various services to solve the problems the model can't do on its own. Make it feel smart rather than be smart. This means that a…

You can tell it "be careful about the tokenizer issues" in Swedish and see how that changes the behavior.

The only thing that this stupid test demonstrates is that LLM metacognitive skills are still lacking. Which shouldn't be a surprise to anyone. The only surprising thing is that they have metacognitive skills, despite the base model training doing very little to encourage their development.

Re: The Case That A.I. Is Thinking

#892

Post by isaacremuant »

hackinthebochs wrote: post #676
isaacremuant wrote: post #632

Earlier quoted context omitted.

No. Cars don't walk. Submarines don't swim. You can call it navigation if you want but words have meaning. > to move through water by moving the body or parts of the body.

Spinning propellers is "moving parts of the [submarines] body"

No they aren't. Of course you cans also call it's sonar eyes but it isn't.

Anthropomorphizing cars doesn't make them humans either.

Re: The Case That A.I. Is Thinking

#893

Post by adamzwasserman »

I've written a full response to Somers' piece: The Case That A.I. Is Thinking: What The New Yorker Missed: https://emusings.substack.com/p/the-case-that-ai-is-thinking...

The core argument: When you apply the same techniques (transformers, gradient descent, next-token prediction) to domains other than language, they fail to produce anything resembling "understanding." Vision had a 50+ year head start but LLMs leapfrogged it in 3 years. That timeline gap is the smoking gun.

The magic isn't in the neural architecture. It's in language itself—which exhibits fractal structure and self-similarity across scales. LLMs navigate a pre-existing map with extraordinary regularity. They never touch the territory.

Re: The Case That A.I. Is Thinking

#894

Post by lossyalgo »

nxor wrote: post #764
lossyalgo wrote: post #698

Earlier quoted context omitted.

Tell that to German-speakers, where the verb comes last, and the order of things in sentences is not anything like English, therefore requiring you to think of the entire sentence before you just spit it out. Even the numbers are backwards (twenty-two is two-and-twenty) which requires thinking. Furthermore, when you ask an LLM to count how many r's are in the word strawberry, it will give you a random answer, "think"…

The part about strawberry is just not right. That problem was solved. And I do think it's a stretch to say German speakers think of the entire sentence before speaking it.

LLMs were not designed to count letters[0] since they work with tokens, so whatever trick they are now doing behind the scenes to handle this case, can probably only handle this particular case. I wonder if it's now included in the system prompt. I asked ChatGPT and it said it's now using len(str) and some other python scripts to do the counting, but who knows what's actually happening behind the scenes.

[0] https://arxiv.org/pdf/2502.16705

Re: The Case That A.I. Is Thinking

#895

Post by popalchemist »

TheOtherHobbes wrote: post #606
popalchemist wrote: post #553

Earlier quoted context omitted.

without getting into theory of mind it's a bit difficult to elaborate, and I don't have the time or the will for that. But the short version is that thinking is interconnected with BEING as well as will, and the Agent has neither, in a philosophically formal sense. The agent is deterministically bound. So it is a fancy Rube Goldberg machine that outputs letters in a way that creates the impression of thought, but it…

The obvious counterargument is that a calculator doesn't experience one-ness, but it still does arithmetic better than most humans. Most people would accept that being able to work out 686799 x 849367 is a form of thinking, albeit an extremely limited one. First flight simulators, then chess computers, then go computers, then LLMs are the same principle extended to much higher levels of applicability and complexity.…

Some of your points are lucid, some are not. For example, an LLM does not "work out" any kind of math equation using anything approaching reasoning; rather it returns a string that is "most likely" to be correct using probability based on its training. Depending on the training data and the question being asked, that output could be accurate or absurd.

That's not of the same nature as reasoning your way to an answer.

Re: The Case That A.I. Is Thinking

#896

Post by ACCount37 »

procaryote wrote: post #868
ACCount37 wrote: post #583

Earlier quoted context omitted.

Again, you're taking a shortcut. "Markov chain" as an excuse to declare "no intelligence". It would be much more honest to say "LLMs are not intelligent because I don't want them to be". Would also explain why you overlook the ever-mounting pile of tasks that were thought to require intelligence, and that LLMs now casually beat an average (presumably intelligent) human at.

If you go around believing all things no one has disproved yet, you will have a very busy belief system.

By now, if you claim "LLMs aren't thinking", the burden of proof is on you. They obviously and openly do.

The pile of capabilities that were previously exclusive to human intelligence is ever growing - and the amount of claims of "LLMs can't do X" that are followed up by examples of LLMs doing X shortly after is staggering.

Re: The Case That A.I. Is Thinking

#897

Post by solumunus »

emp17344 wrote: post #887
solumunus wrote: post #849

Earlier quoted context omitted.

If the LLM output is more effective than a human at problem solving, which I think we can all agree requires intelligence, how would one describe this? The LLM is just pretending to be more intelligent? At a certain point saying that will just seem incredibly silly. It’s either doing the thing or it’s not, and it’s already doing a lot.

LLM output is in no way more effective than human output.

An LLM can absolutely solve programming problems better than some humans. There is plenty of human programmer output that is worse than what an LLM produces, LLM’s can find bugs that weak coders can’t. There are human beings in this world who could dedicate their life to programming and could never be better than an LLM. Do you dispute any of this?

Re: The Case That A.I. Is Thinking

#898

Post by danielmarkbruce »

n0w wrote: post #890
danielmarkbruce wrote: post #882

Earlier quoted context omitted.

but nobody is using LLMs all by themselves. Long-term memory is stored outside the model. In fact, Andrej Karpathy recently talked about the idea that it would be great if we could get LLMs to not know any facts, and that humans poor memory might be a feature which helps with generalization rather than a bug.

This is an interesting idea. I wonder if it's more that we have different "levels" of memory instead of generally "poor" memory though. I'm reminded of an article on the front page recently about the use of bloom filters for search. Would something like a bloom filter per-topic make it easier to link seemingly unrelated ideas?

The details of the search are just details. The way current "LLM Apps" like chatgpt work are just fine. They access material from various places, put it in the context, the LLM reacts.

Re: The Case That A.I. Is Thinking

#899

Post by solumunus »

staticman2 wrote: post #867
solumunus wrote: post #850

Earlier quoted context omitted.

The trouble is that no one knows enough about how the brain works to refute that claim.

There's no serious claim that needs refuting. I don't think any serious person thinks LLMs work like the human brain. People claiming this online aren't going around murdering their spouses like you'd delete an old LLama model from your hard drive. I'm not sure why people keep posting these sorts of claims they can't possibly actually believe if we look at their demonstrable real life behavior.

We’re obviously more advanced than an LLM, but to claim that human beings simply generate output based on inputs and context (environment, life experience) is not silly.

> People claiming this online aren't going around murdering their spouses like you'd delete an old LLama model from your hard drive.

Not sure what you’re trying to say here.

Re: The Case That A.I. Is Thinking

#900

Post by neom »

hodgehog11 wrote: post #841
neom wrote: post #839

I wrote about this the other day more fully. I'd suspect sooner rather than later we formalize consciousness as self model coherence. Simply any dynamical state where predictive and reflective layers remain mutually consistent. Machines will exhibit that state, and for operational purposes it will count as consciousness. Philosophers will likely keep arguing, but it makes sense for industry and law to adopt something…

Consistency is one aspect, but it is not enough. I believe (and this is somewhat based in other arguments from neuroscience and discussions with alignment researchers) that two more are necessary: compression, which demonstrates algorithmic development; and linear representation capacity, as this is the only way that we really interpret the world, and therefore will only define another as intelligent if it can distil…

I think compression is probably a natural consequence of coherent self models? Isn't requiring other minds to package their intelligence in human interpretable linear narratives is like requiring dolphins to demonstrate intelligence through written language?
Post reply on HN