Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I don't know how you get here from “predict the next word”

grumpy-economist.com

Post reply on HN
Print view View original

271–275 of 275 posts

Re: I don't know how you get here from “predict the next word”

#271

Post by qsera »

basch wrote: post #270
qsera wrote: post #260

Earlier quoted context omitted.

>No, it's not equivalent to nested if statements. It is. If you control the randomness involved, the output of a model is completely deterministic. Which means that it can be represented by a huge lookup table. Anything that can be represented by a lookup table can be expressed as an `if then else` statement.

That’s not true in practice thought https://sulbhajain.medium.com/why-llms-arent-truly-determini...

that is why I said

>If you control the randomness involved

Re: I don't know how you get here from “predict the next word”

#272

Post by fc417fc802 »

basch wrote: post #270
qsera wrote: post #260

Earlier quoted context omitted.

>No, it's not equivalent to nested if statements. It is. If you control the randomness involved, the output of a model is completely deterministic. Which means that it can be represented by a huge lookup table. Anything that can be represented by a lookup table can be expressed as an `if then else` statement.

That’s not true in practice thought https://sulbhajain.medium.com/why-llms-arent-truly-determini...

Even if it were true he would still be wrong. I wish HN had some mechanic to evict these sorts of troll accounts that attempt to score rhetorical points rather than honestly learning about a subject. The usual vote and flag mechanics don't work here because there's no singular and overt violation of the guidelines taking place.

Re: I don't know how you get here from “predict the next word”

#273

Post by throw310822 »

sasjaws wrote: post #266
throw310822 wrote: post #180

Earlier quoted context omitted.

> this might be misleadingly interpreted as an LLM having "thought out an answer" I'm convinced that that is exactly what happens. Anthropic confirms it: "Claude will plan what it will say many words ahead, and write to get to that destination. We show this in the realm of poetry, where it thinks of possible rhyming words in advance and writes the next line to get there. This is powerful evidence that even though mod…

This is about reasoning tokens right? I didnt mean that, nanogpt doesnt do that. Nanogpt inference just outputs letters directly, no intermediate tokens.

No, this is about normal tokens. While a SOTA LLM outputs a token at a time, it already has a high level plan of what it is going to say many tokens ahead. This is in reply to the GP who thinks that an LLM can somehow produce coherent and thoughtful sentences while never seeing more than one token ahead.

Re: I don't know how you get here from “predict the next word”

#274

Post by basch »

qsera wrote: post #271
basch wrote: post #270

Earlier quoted context omitted.

That’s not true in practice thought https://sulbhajain.medium.com/why-llms-arent-truly-determini...

that is why I said >If you control the randomness involved

Which you essentially cannot do

It is inherently randomized

Re: I don't know how you get here from “predict the next word”

#275

Post by qsera »

basch wrote: post #274
qsera wrote: post #271

Earlier quoted context omitted.

that is why I said >If you control the randomness involved

Which you essentially cannot do It is inherently randomized

But isn't the link shared by that comment doing exactly that

https://sulbhajain.medium.com/why-llms-arent-truly-determini...

>The Thinking Machines research team showed it’s possible to fix this. They built batch-invariant kernels for RMSNorm, matrix multiplication, and attention, integrating them into the open-source inference engine vLLM.

>The outcome: 1,000 identical prompts, 1,000 identical outputs. Perfect reproducibility.

??

Post reply on HN