Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I'm still bearish on LLMs after Navier-Stokes

dank.systems

Post reply on HN
Print view View original

151–160 of 642 posts

Re: Why I'm still bearish on LLMs after Navier-Stokes

#151

Post by willmarch »

zug_zug wrote: post #145
willmarch wrote: post #86

Earlier quoted context omitted.

Yes, this is essentially how AlphaGo and AlphaZero algorithms work to train superhuman Go/chess/shogi agents. It’s an elegant algorithm that is analogous to how humans learn games.

Well except AlphaZero played 44 million chess games in that time (and actually played with a 44 core computer). So I'd like to point out that the human is still just a few orders of magnitude more efficient.

Yes, we all know that biological systems are more efficient than machines through billions of years of evolution and natural selection but the overall process is largely the same (interacting with an environment, learning from results, improving underlying architecture, etc); efficiencies will come with more time and improvements.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#152

Post by skydhash »

willmarch wrote: post #112
what wrote: post #107

Earlier quoted context omitted.

> considering LLMs currently play better than a brand new human player would They’ve ingested all the literature on playing chess, a brand new human player has not.

Yes, but my point is that humans can’t even do the thing that the above comments are claiming humans can do (read a book or two and be decent at chess), and then they complain that LLMs can’t do the same thing (that humans can’t do either). We seem to be moving goalposts to the point that humans don’t even live up to the expectations of the AI critics. The only way you get better at chess is by playing a lot of games…

> The only way you get better at chess is by playing a lot of games and learning from mistakes

How can you play without being aware of the rules and how can you learn from your mistakes without knowing they are mistakes? That’s what I said about reading a book of two. It is to kickstart the process. Then mastery is gained over time through practice.

This kickstarting then gradual refinement is how most people learn. And the foundational knowledge stays. Even a basic player knows to not do illegal moves.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#153

Post by thesmtsolver2 »

consensus1 wrote: post #126
thesmtsolver2 wrote: post #54

Earlier quoted context omitted.

Human beings can play chess directly without coding up a tool.

Very poorly compared to the tools we have built. Similar to the LLM.

Comparing to raw LLMs? Much much better.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#154

Post by svachalek »

manquer wrote: post #63
WhitneyLand wrote: post #59

Earlier quoted context omitted.

1. It’s hard to trust a 2026 paper that’s showing results for such old models. 2. Chess seems to be a poor benchmark for generalized strategic reasoning. People who are good at it rely more on experience and deep domain expertise than on skills that generalize to make them experts at unrelated tasks. 3. The study sounds like proving humans will never fly because they don’t have wings. In reality, humans do fly, and C…

> People who are good at it rely more on experience and deep domain expertise People are good are 1900 or 2100 above and the top ones who spend decades in the field i.e. deep expertise are well in the 2200-2700 range. A 1100 player is none of these things, they are purely relying on strategic reasoning there is a good chance they cannot name a single opening or articulate clearly why a move was appropriate. 1100 is q…

1100 at online speed chess or something, could be. I'm not that deep in the chess world but everyone I know that can make 1100 in official rating can name a dozen openings and most of the known tactics, and is pretty good at applying at least one opening.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#155

Post by arm32 »

handfuloflight wrote: post #67
randomImmigrant wrote: post #11

I think bearish on LLMs for automation, and bullish for LLM+human experts in specific fields, is about the right expectation for current architectures. Apart from issues with task generalization, or perhaps related to it, is the fact that LLMs have real trouble with timekeeping, and cannot estimate the real world time it will take them to do things very well. This plus the memory issues make dreams of long horizon ag…

> This plus the memory issues make dreams of long horizon agents, that could plausibly handle changing specifications, quite implausible with current architectures. Any reason why that can't be solved through context management and keep-forward scaffolding?

Write the same sentence you just wrote back to me, but in only four words and let’s see if it has the same meaning.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#156

Post by famouswaffles »

diehunde wrote: post #149
famouswaffles wrote: post #119

Earlier quoted context omitted.

If all you do is read chess books, you'll be a shit player. Training and practice is what it takes to be great.

Oh right. But if all you do is reading programming books you are an amazing programmer? Where is all the training and practice LLMs did to become so good at coding?

LLMs (and Humans) don't get really good from programming books lol. The training and practice is the actual code they predict and learn from in the process of predicting.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#157

Post by lantry »

arm32 wrote: post #155
handfuloflight wrote: post #67

Earlier quoted context omitted.

> This plus the memory issues make dreams of long horizon agents, that could plausibly handle changing specifications, quite implausible with current architectures. Any reason why that can't be solved through context management and keep-forward scaffolding?

Write the same sentence you just wrote back to me, but in only four words and let’s see if it has the same meaning.

"Any reason why that can't be solved through context management and keep-forward scaffolding?"

becomes

"load bearing context seam"

/s

Re: Why I'm still bearish on LLMs after Navier-Stokes

#158

Post by linkjuice4all »

xdavidliu wrote: post #128
uncivilized wrote: post #98

Earlier quoted context omitted.

HN is no different than Reddit, or any social media for that matter, in that commenters pretend to read articles.

that is if it even a human commenter at all

State-sponsored psyop meta comments aside, the models obviously continue to get better, but there is still a lot of 'guard railing' required to keep even the latest models completely on-task. The chess example is interesting because it's clearly a well-studied and established domain so the rules, strategies, and whatever else is in the training data should make yield excellent results; but clearly there is some behavior in these systems that's difficult to engineer out.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#159

Post by rsfern »

willmarch wrote: post #82
skydhash wrote: post #72

Earlier quoted context omitted.

Pretty much this. Feed it a book or two on chess, and you should have a decent (or good) player. That's the generic intelligence people have. The aims is not to be supremely talented at something, but being able to read a manual and figure how to use/play something. Mastery can be gained overtime.

If you gave a human a book or two on chess they would not become a decent player (they would be closer to 500-600 than 1100 ELO) and they would only get better after playing hundreds or thousands of games (often making illegal moves and moves that violate the rules of chess as they learn). Your assumptions/intuition about generic human intelligence feels quite incorrect, considering LLMs currently play better than a…

the discussion isn’t really about whether language models can become strong chess players though, the point is they seem to struggle to consistently make valid moves. Most humans don’t need to read two books to pick that up, just a couple lines of basic instructions

Re: Why I'm still bearish on LLMs after Navier-Stokes

#160

Post by handfuloflight »

arm32 wrote: post #155
handfuloflight wrote: post #67

Earlier quoted context omitted.

> This plus the memory issues make dreams of long horizon agents, that could plausibly handle changing specifications, quite implausible with current architectures. Any reason why that can't be solved through context management and keep-forward scaffolding?

Write the same sentence you just wrote back to me, but in only four words and let’s see if it has the same meaning.

Remember specifics, keep going.
Post reply on HN