Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I'm still bearish on LLMs after Navier-Stokes

dank.systems

Post reply on HN
Print view View original

561–570 of 642 posts

Re: Why I'm still bearish on LLMs after Navier-Stokes

#561

Post by Capricorn2481 »

thom wrote: post #276
foldr wrote: post #267

Earlier quoted context omitted.

Humans don't need a lot of training and finite tuning to make only legal moves. An intelligent adult could simply read a short summary of the rules of chess and then, if they were careful, play a very bad game of chess without making illegal moves. An LLM that has not been trained on any chess data cannot do that, at present. If you doubt it, take a current model and tell it that you want to play it at a variant of c…

How long a prompt do you think would be required to cajole an LLM into making legal moves at the rate of a human? Or do you think no amount of prompting could do that?

The actual question is backwards: how do we keep the prompt and context small enough so the LLM doesn't start hallucinating basic rules of chess.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#562

Post by d0mine »

0c3ca83 wrote: post #499
reedlaw wrote: post #494

Earlier quoted context omitted.

Because agents lack human judgment. At the very least there's a need for a human-in-the-loop with agentic processes. Otherwise, it's like running a coding harness with --dangerously-skip-permissions all the time.

Why do you think judgement is impossible to automate? What aspects of it do you think make it hard?

At the moment, the hurdle is [missing] context [aka shared model/understanding of the business, common sense].

Though it is not hard to imagine that any and all communications being recorded for AI consumption in the future.

The Next one is the relative lack of prompt feedback (expect the blowup in finite time like Navier-Stokes ;) [there is not much feedback even for humans at middle management positions].

The cost [tokens] might become prohibitive unless LLMs improve further [not a guarantee].

Re: Why I'm still bearish on LLMs after Navier-Stokes

#563

Post by numitus »

MattCruikshank wrote: post #560
numitus wrote: post #529

Earlier quoted context omitted.

I am bad in chess game by itself like 1200 ELO, but I can write Programm and win player with 2600 ELO. Does it mean I am pro chess gamer?

Do I care if Richard Feynman was only able to do nuclear physics with the help of an abacus? Sure, a Spelling Bee is a fun thing to have. Little kids work so hard. They practice for hours. There's joy and heartbreak. Prized, sometimes. Notoriety. But in the real world, computer-assisted spelling is by far the norm. Sometimes you care about the Bee, sometimes you care about the results.

This is called a benchmark. We run a calculation of Pi to evaluate a computer's performance, but we don't allow the script to download a ready-made solution. When we evaluate a runner, we don't let them use a bicycle. When we evaluate a new LLM, we don't allow it to send a request to a team of programmers, so using a chess engine for an LLM is cheating

Re: Why I'm still bearish on LLMs after Navier-Stokes

#564

Post by lolakutty »

moomoo11 wrote: post #545
lolakutty wrote: post #539

Earlier quoted context omitted.

>like i said, if you are still having quality issues in 2026, that's a skill gap. So what ever shit the "AI" gets wrong, it is the user's fault, right?

yes, if you use a tool improperly that's on you. i am specifically talking about people who don't use skills (the most basic shit) and then get sloppy code.

But what if I use the skills, and still can't model to follow it? Does the AI company say anywhere that they will refund the tokens if the model does not follow what is written in the Skills file?

If there is no such guarantee, why should I spend time writing an elaborate skill file? There is no telling when the model chose to ignore stuff in it.

I don't understand how people can work with something like that!

Re: Why I'm still bearish on LLMs after Navier-Stokes

#565

Post by vanuatu »

openfront wrote: post #549
reedlaw wrote: post #481

Earlier quoted context omitted.

In my own experience as a software developer, I see no justification for the hype. Demand for software is practically infinite, and agents aren't mind readers so they'll always need workers to turn needs into prompts. Despite how much work they get done on their own, Parkinson's Law ( https://en.wikipedia.org/wiki/Parkinson%27s_Law ) is still in effect. You could even expand "Work expands so as to fill the time avail…

I think the fear is that softwaring engineering becomes a low-skill profession. If the models get good enough you won't need years of experience to be a decent programmer.

I think its the opposite, the grunt work is done by AI, the remaining skills are difficult to acquire and software engineers command a higher premium (but there may be less software engineers in aggregate)

Re: Why I'm still bearish on LLMs after Navier-Stokes

#566

Post by vanuatu »

Capricorn2481 wrote: post #546
plaidfuji wrote: post #463

This is the most grounded and coherent take I’ve seen on the actual realizable value of LLMs.. pretty much since they came out. > the classes of firms that can accept the use of fully autonomous LLMs are few, by my count just three: 1. those who can accept failure cheaply: firms that would otherwise hire interns, firms involved in rapid prototyping work, etc. 2. those who need done a small set of narrowly defined tas…

I think using AI for customer service is really, really ineffective, and it's plain to anybody that has interacted with it. It's a shame that we've had decades of shitty customer service from companies that have the most responsibility and resources to do it, and people in tech have shrugged their shoulders saying "it's unreasonable to expect people to scale up their customer service! They gotta make money!" Now AI c…

I've seen it be very effective (moreso than human agents at resolving issues) but its extremely implementation sensitive, so you're more likely to encounter a bad one in the wild than a good one. There are voice agents picking up phones that the vast majority of people don't realize is an agent

Re: Why I'm still bearish on LLMs after Navier-Stokes

#567

Post by sthuck »

openfront wrote: post #537
jimmaswell wrote: post #497

Earlier quoted context omitted.

As a programming lead on a hobbyist video game project, we're reaping massive rewards being in category #1. I keep the architecture and important details in check while letting frontier models go ham. As alluded, it's a video game, not a life support system, so bugs are low-impact. But even better, the defect rate is actually the lowest it's ever been. Insidious bugs baked in by years of accumulated human error are t…

I've found the same, the rate of bugs has dropped pretty dramatically after switching to ai generated code. I think it's partly because ai will write 1000s of lines of unit tests without complaining. I also have a github workflow where claude runs the /code-review command on every PR.

In the world of web apps, I find the agent's ability to write good e2e tests to be a real game changer. Turns out with enough rigor you can write pretty stable mostly not flaky e2e tests. And even flaky ones are fixed quickly due to a fuck ton of assertions at every step.

Test code looks like a mess, even more than the usual LLM code. takes a while to let it go. Test report looks beautiful though.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#568

Post by autoexec »

Capricorn2481 wrote: post #546
plaidfuji wrote: post #463

This is the most grounded and coherent take I’ve seen on the actual realizable value of LLMs.. pretty much since they came out. > the classes of firms that can accept the use of fully autonomous LLMs are few, by my count just three: 1. those who can accept failure cheaply: firms that would otherwise hire interns, firms involved in rapid prototyping work, etc. 2. those who need done a small set of narrowly defined tas…

I think using AI for customer service is really, really ineffective, and it's plain to anybody that has interacted with it. It's a shame that we've had decades of shitty customer service from companies that have the most responsibility and resources to do it, and people in tech have shrugged their shoulders saying "it's unreasonable to expect people to scale up their customer service! They gotta make money!" Now AI c…

AI is perfect for tasks if you don't care about them being done correctly. Many companies just don't care about providing effective customer service and really don't want to provide it at all.

Re: Why I'm still bearish on LLMs after Navier-Stokes

#569

Post by Capricorn2481 »

moomoo11 wrote: post #519
StevePrefontain wrote: post #423

Earlier quoted context omitted.

Very hard to tell if this is satire. It reads like one of those old r/programmerhumor posts where people competed to come up with the worst designs possible for a volume control. I could not think of a worse technology to use for an ETL pipeline than throwing LLMs at it and asking it to vibe out the correctness of the data every time it runs.

lol. wish it was. fwiw i was pretty much bearish on AI for years because I did go deep into this shit since i got access to GPT 3. but since astra i have been quite bullish. 98% of people think AI means what gemini tells them when they do a google search. of that 2% who go beyond... maybe 20% of those are using AI to code. most SWE still think "using AI" to code means the copilot pane they open on the side in VSC. th…

> 98% of people think AI means what gemini tells them when they do a google search.

> of that 2% who go beyond... maybe 20% of those are using AI to code.

> so of that 20%, maybe 5% have... maybe 1-5% of that 5% actually went deeper

How can you write any of this when you said you just started liking LLMs with Astra, a model that released two weeks ago. You're whipping out a bunch of made up stats trying to describe large swaths of programmers while admittedly being unfamiliar with the capabilities until two weeks ago? Why are you so antagonistic towards people's skepticism if you used them for years and found no value in them? This sounds more like a comment trying to generate massive amounts of FOMO. "If you REALLY put a lot of your money into them, that's when they shine! Don't believe the haters!"

> like i said, if you are still having quality issues in 2026, that's a skill gap.

> also, agents now continually improve the process.

> for a business, the only thing that matters is transaction log. for 99% of businesses, swe are a cost.

Maybe you should try building something for more than two weeks before making such big declarations, because this really does read like someone who is excited about a novel they wrote last night at 3 am

Re: Why I'm still bearish on LLMs after Navier-Stokes

#570

Post by thinkharderdev »

Analemma_ wrote: post #523
noir_lord wrote: post #507

Earlier quoted context omitted.

It requires general intelligence and we don't even have a good understanding of how our's works or a particularly good way of quantifying it. The counter argument is of course maybe you don't need to understand our kind of intelligence to create a different kind and that could well be true but then how do you determine if a system is intelligent. Unless the new system is intelligent enough to reason with us on our le…

Why do you think it requires general intelligence? The parent and I aren't being obtuse here: the history of artificial intelligence research is littered with examples of humans confidently declaring that task X requires general intelligence, then getting humiliated by a neural network doing task X better than humans a few years later. See Go, driving, art (you can complain about the quality of AI art, but it's winni…

I'm not entirely sure "general" intelligence is a meaningful concept but I take judgement to mean something like "the ability to make decisions under uncertainty without direct feedback." People seem to be able to do this using a lot of context and tacit knowledge. It's not really clear how you hill-climb to that capability through RL.

> the history of artificial intelligence research is littered with examples of humans confidently declaring that task X requires general intelligence, then getting humiliated by a neural network doing task X better than humans a few years later

True, but the history of AI research is also littered with AI researchers confidently predicting X job will be replaced by AI and being completely wrong because they don't actually understand what those jobs actually are. See Geoff Hinton predicting that Radiologists would be obsolete by ~2020, or predictions that truck drivers would all be replaced by self-driving tech.

Post reply on HN