Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I'm still bearish on LLMs after Navier-Stokes

dank.systems

Post reply on HN
Print view View original

501–510 of 642 posts

Re: Why I'm still bearish on LLMs after Navier-Stokes

#501

Post by glitchc »

well_ackshually wrote: post #458
glitchc wrote: post #443

Earlier quoted context omitted.

> Writing a well understood engine for a super popular problem does not count as reasoning about the problem. Doesn't writing the engine imply understanding about the problem domain? Tool use is a widely accepted measure of intelligence.

>Tool use is a widely accepted measure of intelligence. Stop anthropomorphizing the parrot. The parrot has had stockfish's source code blasted at high pressure into its head along with dozens of millions of other pieces of code whose sole role is to have efficient algorithms to more or less brute force through the best result. Brute forcing (no matter how smart it is) isn't understanding the problem domain.

I'm not fully convinced that human beings aren't stochastic parrots. Much of the behaviour I observe in daily life reflects a blind adherence to set of beliefs that are an amalgation of "a person of authority said to do this" at an early age.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#502

Post by abletonlive »

plaidfuji wrote: post #463

This is the most grounded and coherent take I’ve seen on the actual realizable value of LLMs.. pretty much since they came out. > the classes of firms that can accept the use of fully autonomous LLMs are few, by my count just three: 1. those who can accept failure cheaply: firms that would otherwise hire interns, firms involved in rapid prototyping work, etc. 2. those who need done a small set of narrowly defined tas…

[flagged]

Re: Why I'm still bearish on LLMs after Navier-Stokes

#503

Post by bhelkey »

sobellian wrote: post #83
joefourier wrote: post #31

Earlier quoted context omitted.

> current frontier models > Gemini 2.5 Pro, O3, Claude Sonnet 3.7 and ChatGPT 4.1 The gap in capabilities between those models which they tested, and actual current frontier ones is enormous. I would not trust that any conclusions they made are applicable.

I tested both myself and a weak bot against Astra xhigh, https://lichess.org/study/27lCQqDa . It's still pretty bad at chess, though it takes longer to devolve into illegal moves.

It looks like it played a fully legal game of chess with one exception, it said "rxd1+" (Rook takes D1 with check) instead of "rd1+" (Rook to D1 with check) on move 29.

I would say this did a really good job of playing chess. It moved the pieces consistently and traded pieces when required.

This is worlds away from the frontier ~1 year ago where models would hallucinate pieces into existence.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#504

Post by Onavo »

sfn42 wrote: post #285
Onavo wrote: post #202

Earlier quoted context omitted.

> even if I give them literal infinite time and all the subagents and internet access.. Don't use the word infinite in any CS claims. They can recreate or approximate monte Carlo tree search and it technically is still a correct solution in your framing of the problem so long they defeat you.

[flagged]

Give me a proof they don't. Because from my observations they clearly do.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#505

Post by Zambyte »

sp527 wrote: post #438
Zambyte wrote: post #374

Earlier quoted context omitted.

Traditional non-chaotic systems* LLMs are deterministic. They are chaotic, which people confuse for non-deterministic.

> They are chaotic, which people confuse for non-deterministic. That's an odd argument, because a lot of people who have struggled to decipher complex chaotic systems would tell you this is a distinction without much of a difference.

It's at least important from philosophical perspective regarding whether LLMs have free will or not. It's hard to say whether or not a human thought or action is driven by free will, people have debated it for centuries. We can provide the same input to an LLM and get the same output, because they are deterministic. Surely it is obvious that a pure function does not have free will, no matter how expensive it is to compute.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#506

Post by sobellian »

bhelkey wrote: post #503
sobellian wrote: post #83

Earlier quoted context omitted.

I tested both myself and a weak bot against Astra xhigh, https://lichess.org/study/27lCQqDa . It's still pretty bad at chess, though it takes longer to devolve into illegal moves.

It looks like it played a fully legal game of chess with one exception, it said "rxd1+" (Rook takes D1 with check) instead of "rd1+" (Rook to D1 with check) on move 29. I would say this did a really good job of playing chess. It moved the pieces consistently and traded pieces when required. This is worlds away from the frontier ~1 year ago where models would hallucinate pieces into existence.

You can still see undercurrents of its old self, once I pointed out the illegal notation it hallucinated prior illegal moves. But I agree, it's leagues apart from prior iterations. It also knew thematic moves in the opening. But whenever it needs to play concretely rather than "I know so-and-so is a good move in these types of positions" it crumbles.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#507

Post by noir_lord »

0c3ca83 wrote: post #499
reedlaw wrote: post #494

Earlier quoted context omitted.

Because agents lack human judgment. At the very least there's a need for a human-in-the-loop with agentic processes. Otherwise, it's like running a coding harness with --dangerously-skip-permissions all the time.

Why do you think judgement is impossible to automate? What aspects of it do you think make it hard?

It requires general intelligence and we don't even have a good understanding of how our's works or a particularly good way of quantifying it.

The counter argument is of course maybe you don't need to understand our kind of intelligence to create a different kind and that could well be true but then how do you determine if a system is intelligent.

Unless the new system is intelligent enough to reason with us on our level in a way we can "see" is intelligent it becomes a philosophical argument.

We also have a natural inclination towards anthropomorphising systems that mirror us, this is already a problem with LLM's and people overestimating their capabilities or forming actual emotional attachment.

Then there are those of us who know more about how they work who in theory should be more immune to that and aren't.

I added some stuff to my agent.md to make it sound less human and to communicate more like the machine be it is because I find the faked emotion extremely jarring.

It can't be sorry, it's a set of numbers, it sits in the linguistic uncanny valley.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#508

Post by willmarch »

skydhash wrote: post #298
willmarch wrote: post #171

Earlier quoted context omitted.

Reading can kickstart the process, but you can also make random moves guided by some sort of system (such as a computer GUI) or learn by watching other players play. The overall point is that you learn through observation and lots of trial and error (whether you are a human or a computer). And beginners in chess often make illegal moves even after learning the rules, it's fairly common. It feels like you're trying to…

> The overall point is that you learn through observation and lots of trial and error That’s the most inefficient way and people usually avoid doing that. Instead they find someone that knows how to do the thing and ask him to be a teacher. Or use a proxy like a book or videos. > It feels like you're trying to say that humans never make illegal moves while learning chess, which doesn't match with my experience. I'm t…

The only way to master anything is lots of trial and error. Coaches and teachers can help guide you towards more focused trial and error paths but the student still has to do the lessons and put in the work of learning, and learning only truly happens through doing.

Humans are not perfect and make mistakes in learning even when they have memorized the rules. A simple example is new players will often move a piece, exposing their king to check, and a more experienced player must point out to them that they have made an illegal move (because a new player often has not encoded that pattern for looking for exposed checks because they're more focused on how the pieces move, not what that piece exposes.)

We're just going to have to agree to disagree here.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#509

Post by snapetom »

0c3ca83 wrote: post #489
reedlaw wrote: post #481

Earlier quoted context omitted.

In my own experience as a software developer, I see no justification for the hype. Demand for software is practically infinite, and agents aren't mind readers so they'll always need workers to turn needs into prompts. Despite how much work they get done on their own, Parkinson's Law ( https://en.wikipedia.org/wiki/Parkinson%27s_Law ) is still in effect. You could even expand "Work expands so as to fill the time avail…

Why do you belive that agents wouldn't be able to take over product management, and generate prompts for the "software engineer" agents?

When done right, product management is 90% idea generation and market/user research. So many companies do it poorly that PdMs are beaten to think their job is to wallow in bureaucracy and metrics.

LLMs are great at helping with aspects of market research and that’s about it. Aka it’s a good deep search engine. It’s not going to decide what features solve certain customer pain points. It’s certainly not going to prioritize and coordinate between competing stakeholders.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#510

Post by foldr »

thom wrote: post #464
foldr wrote: post #414

Earlier quoted context omitted.

That's what a human will do if they already have a physical board to look at. But if someone, say, posed you a chess exam question via FEN notation, or as a sequence of moves in algebraic notation, you'd sketch a visual representation of the board off your own initiative to help you answer the question. There is nothing in principle to stop the LLM creating its own board representations in whatever format enables it…

I maintain that the amount of effort to teach a human to do this vastly outweighs the amount of effort to teach an LLM to do this unless you're deliberately trying to make them fail. I honestly have no bigger point than that, I just think this isn't a very good thing by which to evaluate LLM capabilities. If there's no argument you'll accept, I am happy to move on.

You don’t need to teach a human anything except the rules of chess and the details of a particular chess notation. No special skill or training is required to make a sketch of a chess board. Surely there is no chess player who, if confronted with a sequence of chess moves in algebraic notation, would not think to construct a representation of the chess board in order to understand what was going on.

> I just think this isn't a very good thing by which to evaluate LLM capabilities

I don’t think any single task is a good way to evaluate LLM capabilities, but I don’t see why chess is worse than a lot of other tasks. (Of course it is of no practical consequence whether LLMs can play chess, so if you are just making that point, then yes, I agree.)

> If there's no argument you'll accept

It’s a little unfair to suggest that I wouldn’t accept any argument whatever for your position just because I haven’t been convinced by your very brief comments so far. I could equally well say the same thing to you!

Post reply on HN