Earlier quoted context omitted.
> just passing electric signals back and forward Ok, feel free to call yourselves a toaster, I don't mind!
What, reductionism only works when you do it?
I didn't
I don't know how you get here from “predict the next word”
161–170 of 275 posts
Re: I don't know how you get here from “predict the next word”
#162A while ago i did the nanogpt tutorial, i went through some math with pen and paper and noticed the loss function for 'predict the next token' and 'predict the next 2 tokens' (or n tokens) is identical. That was a bit of a shock to me so wanted to share this thought. Basically i think its not unreasonable to say llms are trained to predict the next book instead of single token. Hope this is usefull to someone.
I'd like to explore this idea, did you make a blog post about it? is it simple enough to post in the reply?
Look up attention masks
Re: I don't know how you get here from “predict the next word”
#163Earlier quoted context omitted.
As an expert in the field: this is exactly right. LLMs are trained to do whole book prediction, at training time we throw in whole books at the time. It's only when sampling we do one or a few tokens at the time.
Isn't that the same as compressing the whole book , in a special differential format that compares how the text looks from any given point before and after?
There are many ways to model how the model works in simpler terms. Next-word prediction is useful to characterize how you do inference with the model. Maximizing mutual information, compressing, gradient descent, ... are all useful characterisations of the training process.
But as stated above, next token prediction is a misleading frame for the training process. While the sampling is indeed happening 1 token at a time, due to the training process, much more is going on in the latent space where the model has its internal stream of information.
Re: I don't know how you get here from “predict the next word”
#164> the kind of analysis the program is able to do is past the point where technology looks like magic. I don’t know how you get here from “predict the next word.” You're implicitly assuming that what you asked the LLM to do is unrepresented in the training data. That assumption is usually faulty - very few of the ideas and concepts we come up with in our everyday lives are truly new. All that being said, the refine.in…
> You're implicitly assuming that what you asked the LLM to do is unrepresented in the training data. That assumption is usually faulty - very few of the ideas and concepts we come up with in our everyday lives are truly new. I made a cursed CPU in the game 'Turing Complete'; and had an older version of claude build me an assembler for it? Good luck finding THAT in the training data. :-P (just to be sure, I then had…
But the ideas are not 'new'. A benchmark that I use to tell me if an AI is overfitted is to present the AI with a recent paper (especially one like a paxos variant) and have it build that. If it writes general paxos instead of what the paper specified, its overfitted.
Claude 4.5: not overfitted too much -- does the right thing 6/10 times.
Claude 4.6: overfitted -- does the right thing 2/10 times.
OpenAI 5.3: overfitted -- does the right thing 3/10 times.
These aren't perfect benchmarks, but it lets me know how much babysitting I need to do.
My point being that older Claude models weren't overfitted nearly as much, so I'm confirming what you're saying.
Re: I don't know how you get here from “predict the next word”
#165Earlier quoted context omitted.
Statistical models generalize. If you train a model that f(x) = 5 and f(x+1) = 6, the number 7 doesn't have to exist in the training data for the model to give you a correct answer for f(x+2) Similarly, if there are millions of academic papers and thousands of peer reviews in the training data, a review of this exact paper doesn't need to be in there for the LLM to write something convincing. (I say "convincing" rath…
I expected (and still expect) a lot from LLM with cross disciplinary research. I think they should be the perfect tool to find methods or results in a field which look like it could be used in another field.
This might actually be a limitation of the "predict next word" approach since the network is never trained to predict a result in one field from a result in another. It might still make the connection though, but not as easily.
Re: I don't know how you get here from “predict the next word”
#166>I don’t know how you get here from “predict the next word.”
The question puts horse behind the buggy. The main point isn't "from", it is how you get to “predict the next word.” During the training the LLM builds inside itself compressed aggregated representation - a model - of what is fed into it. Giving the model you can "predict the next word" as well as you can do a lot of other things.
For simple starting point for understanding i'd suggest to look back at the key foundational stone that started it all - "sentiment neuron"
https://openai.com/index/unsupervised-sentiment-neuron/
"simply predicting the next character in Amazon reviews resulted in discovering the concept of sentiment.
...
Digging in, we realized there actually existed a single “sentiment neuron” that’s highly predictive of the sentiment value."
Re: I don't know how you get here from “predict the next word”
#167The ideas in the update were previously explored by Gwern 2 years ago: https://www.lesswrong.com/posts/PQaZiATafCh7n5Luf/gwern-s-sh...
Re: I don't know how you get here from “predict the next word”
#168Earlier quoted context omitted.
Semantics. Its a encoded position that represents meaning in a way that is useful and reusable. That is "understanding." It's a mathematical representation of grasp.
Yea, semantics is important. It is not "understanding" any more than a microphone+ADC is hearing.
The distinction you're making reads like substance dualism to me. Are you able to provide a clear and objective metric for assessing "understanding"? If not then you're just handwaving an effectively meaningless semantic distinction.
Re: I don't know how you get here from “predict the next word”
#169Attention is all you need.
[deleted]
Re: I don't know how you get here from “predict the next word”
#170> the kind of analysis the program is able to do is past the point where technology looks like magic. I don’t know how you get here from “predict the next word.” You're implicitly assuming that what you asked the LLM to do is unrepresented in the training data. That assumption is usually faulty - very few of the ideas and concepts we come up with in our everyday lives are truly new. All that being said, the refine.in…
I created a code review pipeline at work with a similar tradeoff and we found the cost is worth it. Time is a non-issue. We could run Claude on our code and call it a day, but we have hundreds of style, safety, etc rules on a very large C++ codebase with intricate behaviour (cooperative multitasking be fun). So we run dozens of parallel CLI agents that can review the code in excruciating detail. This has completely r…
> This has completely replaced human code review for anything that isn't functional correctness
Isn’t functional correctness pretty much the only thing that matters though?