Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I don't know how you get here from “predict the next word”

grumpy-economist.com

Post reply on HN
Print view View original

81–90 of 275 posts

Re: I don't know how you get here from “predict the next word”

#81

Post by jjmarr »

wavemode wrote: post #24

> the kind of analysis the program is able to do is past the point where technology looks like magic. I don’t know how you get here from “predict the next word.” You're implicitly assuming that what you asked the LLM to do is unrepresented in the training data. That assumption is usually faulty - very few of the ideas and concepts we come up with in our everyday lives are truly new. All that being said, the refine.in…

I created a code review pipeline at work with a similar tradeoff and we found the cost is worth it. Time is a non-issue.

We could run Claude on our code and call it a day, but we have hundreds of style, safety, etc rules on a very large C++ codebase with intricate behaviour (cooperative multitasking be fun).

So we run dozens of parallel CLI agents that can review the code in excruciating detail. This has completely replaced human code review for anything that isn't functional correctness but is near the same order of magnitude of price. Much better than humans and beats every commercial tool.

"scaling time" on the other hand is useless. You can just divide the problem with subagents until it's time within a few minutes because that also increases quality due to less context/more focus.

Re: I don't know how you get here from “predict the next word”

#82

Post by friendzis »

Tarq0n wrote: post #31
cyanydeez wrote: post #23

Earlier quoted context omitted.

Economics is the attempt to take sociology and add numbers to make it look like a hard science. The fintechbros then seem to think because they can make numbers go up that this proof it's a hard science.

That's entirely missing the point. "All models are wrong, but some are useful". You can test hypotheses and learn things even about chaotic or emergent systems.

> You can test hypotheses and learn things even about chaotic or emergent systems.

Ah yes, the famous "Cut GDP in half, abolish public schooling and use that as a control" experiment. Majority of economic "models" are entirely correlational without any mechanistic explanation whatsoever or an explanation so superficial that it contradicts either itself or observed reality.

If you look deeper and read explanatory notes of economic laws, the model may refer some publications, but then the actual figures plugged in the model are explained as "these values have been observed to lead to the desired outcomes, therefore are set without any modeling or validation, hope for the best, lesssgoooo".

Re: I don't know how you get here from “predict the next word”

#83

Post by GolfPopper »

chii wrote: post #62
mbgerring wrote: post #39

Earlier quoted context omitted.

People who produced the works LLMs are trained on are not compensated for the value they are now producing, and their skills are increasingly less valued in a world with LLMs. The value the LLMs are producing is being captured by employees of AI companies who are driving up rent in the Bay Area, and driving up the cost of electricity and water everywhere else. Your surprise to people’s objections makes sense if you c…

> People who produced the works LLMs are trained on are not compensated for the value they are now producing the value being extracted via LLM techniques is new value, which did not previously exist. The producer(s) of the old data had an asking price, which was taken by the LLM trainers. They cannot make the argument that since the LLM is producing new value, they should retroactively update their old asking price f…

>The producer(s) of the old data had an asking price, which was taken by the LLM trainers.

This is... blatantly untrue?

https://arstechnica.com/tech-policy/2026/02/microsoft-remove...

https://www.theatlantic.com/technology/archive/2025/03/libge...

Re: I don't know how you get here from “predict the next word”

#84

Post by GodelNumbering »

It is probably the first-time aha moment the author is talking about. But under the hood, it is probably not as magical as it appears to be.

Suppose you prompted the underlying LLM with "You are an expert reviewer in..." and a bunch of instructions followed by the paper. LLM knows from the training that 'expert reviewer' is an important term (skipping over and oversimplifying here) and my response should be framed as what I know an expert reviewer would write. LLMs are good at picking up (or copying) the patterns of response, but the underlying layer that evaluates things against a structural and logical understanding is missing. So, in corner cases, you get responses that are framed impressively but do not contain any meaningful inputs. This trait makes LLMs great at demos but weak at consistently finding novel interesting things.

If the above is true, the author will find after several reviews that the agent they use keeps picking up on the same/similar things (collapsed behavior that makes it good at coding type tasks) and is blind to some other obvious things it should have picked up on. This is not a criticism, many humans are often just as collapsed in their 'reasoning'.

LLMs are good at 8 out of 10 tasks, but you don't know which 8.

Re: I don't know how you get here from “predict the next word”

#85

Post by fc417fc802 »

hnfong wrote: post #52
fc417fc802 wrote: post #22

Earlier quoted context omitted.

> but do not have even a theory about how the behavior emerges from among the math Actually we have an awful lot of those. I'm not sure if emergent is quite the right term here. We carefully craft a scenario to produce a usable gradient for a black box optimizer. We fully expect nontrivial predictions of future state to result in increasingly rich world models out of necessity. It gets back to the age old observation…

> black box optimizer This is admission we don't know how it emerges. Sure, we expect the behavior to emerge, but we don't know how.

No, as I said, we have _lots_ of theories about exactly that at various levels of detail. The theories vary based on (at least) the specifics of the loss function being employed to construct the gradient. Giving an overview of that is far beyond the scope of this comment section (but it's well trodden ground so you can just go ask an LLM).

The "black box" bit refers to a generic, interchangeable optimization algorithm that simply makes the number go down (or up or whatever).

There are certainly various details about the internal workings of models that we don't properly understand but a blanket claim about the whole is erroneous.

Re: I don't know how you get here from “predict the next word”

#86

Post by Alex_L_Wood »

libraryofbabel wrote: post #17

I have come to think “predict the next token” is not a useful way to explain how LLMs work to people unfamiliar with LLM training and internals. It’s technically correct, but at this point saying that and not talking about things like RLVR training and mechanistic interpretability is about as useful as framing talking with a person as “engaging with a human brain generating tokens” and ignoring psychology. At least A…

“Stochastic parrots” only stopped because AI fanboys stopped screaming “AGI” and “it will replace everyone”. Maybe they finally got the memo?

Re: I don't know how you get here from “predict the next word”

#87

Post by modeless »

It's clear that in the general case "predict the next word" requires arbitrarily good understanding of everything that can be described with language. That shouldn't be mysterious. What's mysterious is how a simple training procedure with that objective can in practice achieve that understanding. But then again, does it? The base model you get after that simple training procedure is not capable of doing the things described in the article. It is only useful as a starting point for a much more complex reinforcement learning procedure that teaches the skills an agent needs to achieve goals.

RL is where the magic comes from, and RL is more than just "predict the next word". It has agents and environments and actions and rewards.

Re: I don't know how you get here from “predict the next word”

#88

Post by selridge »

halyconWays wrote: post #65
ChaitanyaSai wrote: post #43

The whole next word thing is interesting isn't it. I like to see it with Dennett's "Competence and comprehension" lens. You can predict the next word competently with shallow understanding. But you could also do it well with understanding or comprehension of the full picture. A mental model that allows you to predict better. Are the AIs stumbling into these mental models? Seems like it. However, because these are suc…

Searle's Chinese Room experiment but without knowing what's in the room, and when you try to peek in you just see a cloud of fog and are left to wonder if it's just a guy with that really big dictionary or something more intelligent.

It's an octopus, perhaps: https://aclanthology.org/2020.acl-main.463.pdf

There's also this blog post: https://julianmichael.org/blog/2020/07/23/to-dissect-an-octo... (which IMO is better to read than the paper)

Re: I don't know how you get here from “predict the next word”

#89

Post by [deleted] »

red75prime wrote: post #68
wavemode wrote: post #41

Earlier quoted context omitted.

Statistical models generalize. If you train a model that f(x) = 5 and f(x+1) = 6, the number 7 doesn't have to exist in the training data for the model to give you a correct answer for f(x+2) Similarly, if there are millions of academic papers and thousands of peer reviews in the training data, a review of this exact paper doesn't need to be in there for the LLM to write something convincing. (I say "convincing" rath…

I think the relevant question is: can a statistical model (or a transformer, in particular) generalize to general reasoning ability?

[deleted]

Re: I don't know how you get here from “predict the next word”

#90

Post by brookst »

measurablefunc wrote: post #55
brookst wrote: post #44

Earlier quoted context omitted.

Not the person you asked, and “novel” is a minefield. What’s the last novel anything , in the sense you can’t trace a precursor or reference? But.. I recently had a LLM suggest an approach to negative mold-making that was novel to me. Long story, but basically isolating the gross geometry and using NURBS booleans for that, plus mesh addition/subtraction for details. I’m sure there’s prior art out there, but that’s tr…

I don't know, that's why I asked b/c I always see a lot of empty platitudes when it comes to LLM praise so I'm curious to see if people can actually back up their claims. I haven't done any 3D modeling so I'll take your word for it but I can tell you that I am working on a very simple interpreter & bytecode compiler for a subset of Erlang & I have yet to see anything novel or even useful from any of the coding assist…

It’s taken me a while to get good at using them.

My advice: ask for more than what you think it can do. #1 mistake is failing to give enough context about goals, constraints, priorities.

Don’t ask “complete this one small task”, ask “hey I’m working on this big project, docs are here, source is there, I’m not sure how to do that, come up with a plan”

Post reply on HN