Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I'm still bearish on LLMs after Navier-Stokes

dank.systems

Post reply on HN
Print view View original

431–440 of 642 posts

Re: Why I'm still bearish on LLMs after Navier-Stokes

#431

Post by gf000 »

datsci_est_2015 wrote: post #401
sailfast wrote: post #380

Earlier quoted context omitted.

What happens when you ask it to play chess against you if the chess game has an API? Are you measuring chess or multi-tasking skill? Also what harness? If you’re using a general harness of course it’s going to try and give you commentary. I say this not because I’m an LLM shill but because false equivalence is all over the place in the space and maybe it’s a fine heuristic for you but probably not a real outcome when…

Why does a 6 year old not need any of these guardrails? Frontier model’s failure modes are a direct refutation of claims that we’ve reached (or will soon reach) the artificial general intelligence. We may have reached an artificial general intelligence, but there may be more complexity to this than even AI thought leaders are talking / influencing about. Maybe not all AGIs have a path to digital singularity. Maybe ou…

Well, would a dissected frontal lobe in and of itself be intelligence?

I think the same goes for LLMs, they may be a core part of an LLM harness, but you may still need a couple other components (e.g. it may itself write itself a deterministic function to validate steps).

In and of itself intelligence is an ill-defined and badly understood concept.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#432

Post by deaton »

MattCruikshank wrote: post #386
carodgers wrote: post #17

This April 2026 paper is a fun and related read. https://arxiv.org/html/2509.24239v4 Researchers asked frontier models to play chess. Have a look at the MAR rates in Table 3. When not explicitly told which moves were legal, no model identified legal moves at a rate better than 80%. Many asked for more illegal moves than legal moves. And even when explicitly told which moves were legal, the models continued to ask for…

What happens when you ask those same frontier models to write a chess-playing program? I feel like, this is a huge stumbling block that many people have. They'll give a model their data, and ask it questions. I vastly prefer letting the model understand the schema, and then writing functions or programs to answer those questions. I feel like I get way, way better answers. I can have it write unit tests for those func…

What would happen if you asked a human developer to write a chess-playing program?

Re: Why I'm still bearish on LLMs after Navier-Stokes

#433

Post by tzone »

carodgers wrote: post #17

This April 2026 paper is a fun and related read. https://arxiv.org/html/2509.24239v4 Researchers asked frontier models to play chess. Have a look at the MAR rates in Table 3. When not explicitly told which moves were legal, no model identified legal moves at a rate better than 80%. Many asked for more illegal moves than legal moves. And even when explicitly told which moves were legal, the models continued to ask for…

It all depends on what prompt you use though. You can just tell all current frontier models to write a chess engine first, and then play a game of chess against you using that engine. It will probably do a pretty good job if you ask it that way (it will also burn a shit ton of tokens, but hey, that is part of the fun).

On that note, I actually had an overall harness (for experimenting) that was essentially like this: "for any task, instead of answering question directly, write a program to answer the question instead. test and verify the program before giving the answer".

It actually worked incredibly well on all "gotcha" LLM questions like math or counting letters in words and all sorts of stuff.

Of course it was ridiculously slow and very expensive but it was a proof of concept that it can actually be much more accurate on every task if you are willing to spend an infinite amount of money.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#434

Post by empath75 »

carodgers wrote: post #17

This April 2026 paper is a fun and related read. https://arxiv.org/html/2509.24239v4 Researchers asked frontier models to play chess. Have a look at the MAR rates in Table 3. When not explicitly told which moves were legal, no model identified legal moves at a rate better than 80%. Many asked for more illegal moves than legal moves. And even when explicitly told which moves were legal, the models continued to ask for…

I want you to consider how relevant this is in any practical sense.

First -- most _people_ cannot do this, without having a physical board in front of them.

Second -- Claude Code is perfectly capable of downloading and running stockfish. People focus too much on LLMs by themselves as the entity of concern instead of the entire harness and all of it's capabilities together.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#435

Post by empath75 »

cbolton wrote: post #417
MattCruikshank wrote: post #386

Earlier quoted context omitted.

What happens when you ask those same frontier models to write a chess-playing program? I feel like, this is a huge stumbling block that many people have. They'll give a model their data, and ask it questions. I vastly prefer letting the model understand the schema, and then writing functions or programs to answer those questions. I feel like I get way, way better answers. I can have it write unit tests for those func…

It's a great test of cognitive abilities. There are many claims that current LLMs surpass humans in cognitive abilities so it's noteworthy that they underperform on that test. Letting the model execute a chess program (that it wrote) would make sense if you're measuring its economic potential, but for cognition that would be cheating just like if you let a human run a chess program. The fact that the human would have…

> It's a great test of cognitive abilities.

It isn't. Stockfish running on your laptop can beat every human being on earth easily at chess. It's not intelligent _at all_ in any sense that matters.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#436

Post by empath75 »

foldr wrote: post #264
BobbyJo wrote: post #118

Earlier quoted context omitted.

I suck at chess. Are you saying I can't be intelligent?

The issue with the models isn't that they play a bad game, but that they persist in making illegal moves. An average intelligent human can be told the rules of chess and then play chess, badly, within the rules.

An average human would have a physical chess board in front of them to remind them of the current state.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#437

Post by MattCruikshank »

lionkor wrote: post #424
MattCruikshank wrote: post #420

Earlier quoted context omitted.

> The fact that the human would have a much harder time writing a useful program is irrelevant. Why? There's a box. You give it a problem, and it comes up with a solution. Why does it matter to you if the box is strictly a LLM, or if the LLM can write code that it executes? Even neater if the box is self-contained with a local model. You provide electricity, and it comes up with solutions. Why does it matter if it ca…

The question is to what end? This is a benchmark task, because playing chess, or solving other well-understood problems is more of a party trick than it is useful. If you let the LLM write a chess program, which it can ONLY do because there are already so many chess programs out there, then the benchmark becomes about recall of popular program source code, not chess.

Do you want to measure the ability of the box, or measure the ability of the box with one hand tied behind its back?

More to my point, I think it's stupid to have LLMs do work that should be done by programs... programs potentially written by LLMs.

I'm advising people that they should think about this distinction, themselves, when they have data and want answers.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#438

Post by sp527 »

Zambyte wrote: post #374
bluegatty wrote: post #114

"are priced according to the narrative that they have produced or will in the very near future produce a fully automated drop-in replacement for most knowledge workers, " No, they're really not. They're priced in a way that would imply AI will be universal form of compute, alongside traditional deterministic systems - which it will be. And that they will capture most of that ... which they won't. The Frontier Labs ar…

Traditional non-chaotic systems* LLMs are deterministic. They are chaotic, which people confuse for non-deterministic.

> They are chaotic, which people confuse for non-deterministic.

That's an odd argument, because a lot of people who have struggled to decipher complex chaotic systems would tell you this is a distinction without much of a difference.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#439

Post by empath75 »

Planktonne wrote: post #254
nimbleal wrote: post #199

Earlier quoted context omitted.

Maybe practically it doesn’t matter? Perhaps AGI is not the model but the model plus everything it’s got access to. If we’re modelling intelligence in the way we seem to have to to have any coherent definition of AGI, it seems to me is always going to be more “intelligent” than alone.

That would mean we should consider any human with coding knowledge a chess grandmaster, which is obviously not the case.

If the goal is merely to "win at chess", then yes, an LLM using stockfish is better than any human alone at performing the task. When you are talking about what AI agents are capable of doing, there is no such thing as "cheating". They are as capable as the tools they can use effectively. The entire history of human civilization was driven by effectively using tools to achieve goals.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#440

Post by datsci_est_2015 »

solenoid0937 wrote: post #411
datsci_est_2015 wrote: post #401

Earlier quoted context omitted.

Why does a 6 year old not need any of these guardrails? Frontier model’s failure modes are a direct refutation of claims that we’ve reached (or will soon reach) the artificial general intelligence. We may have reached an artificial general intelligence, but there may be more complexity to this than even AI thought leaders are talking / influencing about. Maybe not all AGIs have a path to digital singularity. Maybe ou…

Ask a 6 year old to draw a chess board from scratch every turn and they too will make mistakes.

A 6 year old will figure out how to ask you to help them after they get it wrong.
Post reply on HN