Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I don't know how you get here from “predict the next word”

grumpy-economist.com

Post reply on HN
Print view View original

181–190 of 275 posts

Re: I don't know how you get here from “predict the next word”

#181

Post by measurablefunc »

Kim_Bruning wrote: post #127
measurablefunc wrote: post #96

Earlier quoted context omitted.

The workflow is not the issue. You are welcome to try the same challenge yourself if you want. Extra test cases ( https://drive.proton.me/urls/6Z6557R2WG#n83c6DP6mDfc ) & specification ( https://claude.ai/public/artifacts/5581b499-a471-4d58-8e05-1... ). I know enough about compilers, bytecode VMs, parsers, & interpreters to know that this is well within the capabilities of any reasonably good software engineer but th…

Possibly a dumb question: but are you running this in claude code, or an ide, or basically what are you using to allow for iteration?

I'm using Google's antigravity IDE. I initially had it configured to run allowed commands (cargo add|build|check|run, testing shell scripts, performance profiling shell scripts, etc.) so that it would iterate & fix bugs w/ as little intervention from me as possible but all it did was burn through the daily allotted tokens so I switched to more "manual" guidance & made a lot more progress w/o burning through the daily limits.

What I've learned from this experiment is that the hype does not actually live up to the reality. Maybe the next iteration will manage the task better than the current one but it's obvious that basic compiler & bytecode virtual machine design in a language like Rust is still beyond the capabilities of the current coding agents & whoever thinks I'm wrong is welcome to implement the linked specification to see how far they can get by just "vibing".

Re: I don't know how you get here from “predict the next word”

#182

Post by gkbrk »

qsera wrote: post #118
basch wrote: post #111

Earlier quoted context omitted.

Semantics. Its a encoded position that represents meaning in a way that is useful and reusable. That is "understanding." It's a mathematical representation of grasp.

Yea, semantics is important. It is not "understanding" any more than a microphone+ADC is hearing.

A microphone + ADC is hearing though, that's the whole reason we even produce microphones. So that our electronics can hear sound.

Re: I don't know how you get here from “predict the next word”

#183

Post by mekoka »

ChaitanyaSai wrote: post #43

The whole next word thing is interesting isn't it. I like to see it with Dennett's "Competence and comprehension" lens. You can predict the next word competently with shallow understanding. But you could also do it well with understanding or comprehension of the full picture. A mental model that allows you to predict better. Are the AIs stumbling into these mental models? Seems like it. However, because these are suc…

> Are the AIs stumbling into these mental models? Seems like it.

Since nature decided to deprive me of telepathic abilities, when I want to externalize my thoughts to share with others, I'm bound to this joke of a substitute we call language. I must either produce sounds that encode my meaning, or gesture, or write symbols, or basically find some way to convey my inner world by using bodily senses as peripherals. Those who receive my output must do the work in reverse to extract my meaning, the understanding in my message. Language is what we call a medium that carries our meaning to one another's psyche.

LLMs, as their name alludes, are trained on language, the medium, and they're LARGE. They're not trained on the meaning, like a child would be, for instance. Saying that by their sole analysis of the structure and patterns in the medium they're somehow capable of stumbling upon the encoded meaning is like saying that it's possible to become an engineer, by simply mindlessly memorizing many perfectly relevant scripted lines whose meaning you haven't the foggiest.

Yes, on the surface the illusion may be complete, but can the medium somehow become interchangeable with the meaning it carries? Nothing indicates this. Everything an LLM does still very much falls within the parameters of "analyze humongous quantity of texts for patterns with massive amount of resources, then based on all that precious training, when I feed you some text, output something as if you know what you're talking about".

I think the seeming crossover we perceive is just us becoming neglectful in our reflection of the scale and significance of the required resources to get them to fool us.

Re: I don't know how you get here from “predict the next word”

#184

Post by qsera »

gkbrk wrote: post #182
qsera wrote: post #118

Earlier quoted context omitted.

Yea, semantics is important. It is not "understanding" any more than a microphone+ADC is hearing.

A microphone + ADC is hearing though, that's the whole reason we even produce microphones. So that our electronics can hear sound.

So according to you when can you qualify something as capable of hearing

1. Vibrate according input to the sound, is that hearing?

2. Generate electrical signals according to the sound, is that hearing?

3. Amplify electrical signal, does we cross the hearing mark?

4. Record the signal to a cassete tape (or use an ADC -> mp3), are we hearing yet?

5. Play it back through a speaker. Sure, we should be hearing now!

At which point exactly would you say the thing is definitely hearing?

Re: I don't know how you get here from “predict the next word”

#185

Post by kosh2 »

tsunamifury wrote: post #104

I think it’s funny that at Google I invented and productized next word (and next action) predictor in Gmail and hangouts chat and I’ve never had a single person come to me and ask how this all works. To me LLMs are incredibly simple. Next word next sentence next paragraph and next answer are stacked attention layers which identify manifolds and run in reverse to then keep the attention head on track for next token. I…

This is like saying quantum mechanics is really simple to understand, all you have to do is find the right formula and plug in the numbers.

When people talk about understanding, they mean as knowing how the underlying mechanism works often by finding an analog in real life.

Re: I don't know how you get here from “predict the next word”

#186

Post by js8 »

ChaitanyaSai wrote: post #43

The whole next word thing is interesting isn't it. I like to see it with Dennett's "Competence and comprehension" lens. You can predict the next word competently with shallow understanding. But you could also do it well with understanding or comprehension of the full picture. A mental model that allows you to predict better. Are the AIs stumbling into these mental models? Seems like it. However, because these are suc…

Dennett also came to my mind, reading the title, but in a different sense. When people came up with theory of evolution, it was hard to conceive for many people, how do we get from "subtly selecting from random changes" to "build a complex mechanism such as human". I think Dennett offers a nice analogy with a skyscraper, how it can be built if cranes are only so tall?

In a similar way, LLMs build small abstractions, first on words, how to subtly rearrange them without changing meaning, then they start to understand logic patterns such as "If A follows B, and we're given A, then B", and eventually they learn to reason in various ways.

It's the scale of the whole process that defies human understanding.

(Also modern LLMs are not just next word predictors anymore, there is reinforcement learning component as well.)

Re: I don't know how you get here from “predict the next word”

#187

Post by shakna »

jbotz wrote: post #178
shakna wrote: post #99

Earlier quoted context omitted.

Probably worth remembering that ELIZA passed Turing tests, and was the definition of shallow prediction.

ELIZA absolutely did not ever pass anything resembling a real Turing test. A real Turing test is adversarial, the interrogator knows the testees are trying to fool him.

Landauer and Bellman, absolutely put ELIZA to an adversarial Turing test, and called it such, in 1999. [0]

But... Over in 2025, ELIZA was once again, put to the Turing test in adversarial conditions. [1] And still had people think it was a real person, over 27% of the time. Over a quarter of the testees, thought the thing was a human.

The "ELIZA Effect" wasn't coined because everyone understands that an AI isn't conscious.

[0] https://books.google.com.au/books?id=jTgMIhy6YZMC&pg=PA174

[1] https://arxiv.org/html/2503.23674v1

Re: I don't know how you get here from “predict the next word”

#188

Post by Kim_Bruning »

measurablefunc wrote: post #181
Kim_Bruning wrote: post #127

Earlier quoted context omitted.

Possibly a dumb question: but are you running this in claude code, or an ide, or basically what are you using to allow for iteration?

I'm using Google's antigravity IDE. I initially had it configured to run allowed commands (cargo add|build|check|run, testing shell scripts, performance profiling shell scripts, etc.) so that it would iterate & fix bugs w/ as little intervention from me as possible but all it did was burn through the daily allotted tokens so I switched to more "manual" guidance & made a lot more progress w/o burning through the daily…

That's roughly where I'm at too. I have seen people have some more success after having practices though. Possibly the actual workflows needed for full auto are still kind of tacit. Smaller green-field projecs do work for me already though.

Re: I don't know how you get here from “predict the next word”

#189

Post by Kim_Bruning »

withinboredom wrote: post #164
Kim_Bruning wrote: post #144

Earlier quoted context omitted.

> You're implicitly assuming that what you asked the LLM to do is unrepresented in the training data. That assumption is usually faulty - very few of the ideas and concepts we come up with in our everyday lives are truly new. I made a cursed CPU in the game 'Turing Complete'; and had an older version of claude build me an assembler for it? Good luck finding THAT in the training data. :-P (just to be sure, I then had…

But the ideas are not 'new'. A benchmark that I use to tell me if an AI is overfitted is to present the AI with a recent paper (especially one like a paxos variant) and have it build that. If it writes general paxos instead of what the paper specified, its overfitted. Claude 4.5: not overfitted too much -- does the right thing 6/10 times. Claude 4.6: overfitted -- does the right thing 2/10 times. OpenAI 5.3: overfitt…

Could also be that the model has stronger priors wrt Paxos (and thus has Opinions on what good Paxos should look like)

At any rate, with an assembler, you end up with a lot of random letter-salad mnemonics with odd use cases, so that is very likely to tokenize in interesting ways at the very least.

Post reply on HN