Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microgpt

karpathy.github.io

Post reply on HN
Print view View original

151–160 of 354 posts

Re: Microgpt

#151

Post by geokon »

andy12_ wrote: post #147
geokon wrote: post #141

> What’s the deal with “hallucinations”? The model generates tokens by sampling from a probability distribution. It has no concept of truth, it only knows what sequences are statistically plausible given the training data. Extremely naiive question.. but could LLM output be tagged with some kind of confidence score? Like if I'm asking an LLM some question does it have an internal metric for how confident it is in its…

The model could report the confidence of its output distribution, but it isn't necessarily calibrated (that is, even if it tells you that it's 70% confident, it doesn't mean that it is right 70% of the time). Famously, pre-trained base models are calibrated, but they stop being calibrated when they are post-trained to be instruction-following chatbots [1]. Edit: There is also some other work that points out that chat…

In absolute terms sure, but the token stream's confidence changes as it's coming out right? Consumer LLMs typically have a lot window dressing. My sense is this encourages the model to stay on-topic and it's mostly "high confidence" fluff. As it's spewing text/tokens back at you maybe when it starts hallucinating you'd expect a sudden dip in the confidence?

You could color code the output token so you can see some abrupt changes

It seems kind of obvious, so I'm guessing people have tried this

Re: Microgpt

#155

Post by foxglacier »

darkpicnic wrote: post #55
jimbokun wrote: post #47

It’s pretty staggering that a core algorithm simple enough to be expressed in 200 lines of Python can apparently be scaled up to achieve AGI. Yes with some extra tricks and tweaks. But the core ideas are all here.

LLMs won’t lead to AGI. Almost by definition, they can’t. The thought experiment I use constantly to explain this: Train an LLM on all human knowledge up to 1905 and see if it comes up with General Relativity. It won’t. We’ll need additional breakthroughs in AI.

That's an assertion, not a thought experiment. You can't logically reach the conclusion ("It won't") by thinking about it. But it doesn't sound so grand if you say "The assertion I use constantly to explain this".

Re: Microgpt

#157

Post by la_fayette »

This guy is so amazing! With his video and the code base I really have the feeling I understand gradient descent, back propagation, chain rule etc. Reading math only just confuses me, together with the code it makes it so clear! It feels like a lifetime achievement for me :-)

Re: Microgpt

#158

Post by Sammi »

lukan wrote: post #109
hackersk wrote: post #83

[flagged]

"The math makes so much more sense when you implement it yourself vs reading papers." Something I found to be universal true when dealing with math. My brain pretty much refuses to learn abstract math concepts in theory, but applying them with a practical problem is a very different experience for me (I wish school math would have had a bigger focus on practical applications).

It's like you learn math best with your hands. The mind catches up to your hands afterwards.

Re: Microgpt

#159

Post by alexwebb2 »

Lionga wrote: post #146
geokon wrote: post #141

> What’s the deal with “hallucinations”? The model generates tokens by sampling from a probability distribution. It has no concept of truth, it only knows what sequences are statistically plausible given the training data. Extremely naiive question.. but could LLM output be tagged with some kind of confidence score? Like if I'm asking an LLM some question does it have an internal metric for how confident it is in its…

The LLM has an internal "confidence score" but that has NOTHING to do with how correct the answer is, only with how often the same words came together in training data. E.g. getting two r's in strawberry could very well have a very high "confidence score" while a random but rare correct fact might have a very well a very low one. In short: LLM have no concept, or even desire to produce of truth

Huge leap there in your conclusion. Looks like you’re hand-waving away the entire phenomenon of emergent properties.
Post reply on HN