Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

TL;DR of Deep Dive into LLMs Like ChatGPT by Andrej Karpathy

anfalmushtaq.com

Post reply on HN
Print view View original

51–60 of 91 posts

Re: TL;DR of Deep Dive into LLMs Like ChatGPT by Andrej Karpathy

#51

Post by implmntatio »

thomasahle wrote: post #48

I find Meta’s approach to hallucinations delightfully counter intuitive. Basically they (and presumably OpenAI and others): - Extract a snippet of training data. - Generate a factual question about it using Llama 3. - Have Llama 3 generate an answer. - Score the response against the original data. - If incorrect, train the model to recognize and refuse incorrect responses. In a way this is obvious in hindsight, but i…

> but it goes against ML engineers natural tendency when detecting a wrong answer: Teaching the model the right answer.

Hard to buy.

If a machine makes a mistake, it's because it was configured wrong or because of wear and tear, solar flares or some quake or some manufacturing defect in a part. If a learning machine makes a mistake, it's because it's learning has not extended it's rule set to cover that matrix/mistake/pattern, yet; and so it includes that mistake/matrix and other mistakes, analyses for patterns and then creates mistakes that fall into that pattern. Later doing that in a rolling release or canine kind of way and even later learning machines will do it all live, synchronous to their concurrent actions.

But yeah, thinking about that, I see why ML engineers wouldn't get there from scratch. It's a rhythm, after all, an epiphany about or realization of how ones dog, ones brain works, learned and then coded step by step. And there is, of course the variety of how people learn and "realize".

Someone has to show us the work of those savant programmers/engineers I still haven't seen a documentary of.

Re: TL;DR of Deep Dive into LLMs Like ChatGPT by Andrej Karpathy

#52

Post by 2-3-7-43-1807 »

thomasahle wrote: post #48

I find Meta’s approach to hallucinations delightfully counter intuitive. Basically they (and presumably OpenAI and others): - Extract a snippet of training data. - Generate a factual question about it using Llama 3. - Have Llama 3 generate an answer. - Score the response against the original data. - If incorrect, train the model to recognize and refuse incorrect responses. In a way this is obvious in hindsight, but i…

> In a way this is obvious in hindsight, but it goes against ML engineers natural tendency when detecting a wrong answer: Teaching the model the right answer.

But the answer space for LLMs is infinite and unbounded. So, no effort will be complete and you will always end up with the question of how to deal with uncertainty.

But I admit this is a bit of hindsight 20/20.

Re: TL;DR of Deep Dive into LLMs Like ChatGPT by Andrej Karpathy

#53

Post by 2-3-7-43-1807 »

albert_e wrote: post #12

OT: What is a good place to discuss the original video -- once it has dropped out of the HN front-page? I am going through the video myself -- roughly halfway through -- and have a fw things to bring up. Here they are now that we have a fresh opportunity to discuss: 1 - MATH and LLMs I am curious why many of the examples Andrej chose to pose to the LLM were "computational" questions -- for instance "what is 2+2" or s…

> for instance "what is 2+2" or some numerical puzzles that needed algebraic thinking

there is only one algebraic approach to solving something like 2+2 and that is counting! 2+2 = (((0 + 1) + 1) + 1) + 1). but llms are infamously bad at counting. which is why 2+2 isn't an algebraic problem to an llm. it's pattern matching or linguistic reasoning token by token.

Re: TL;DR of Deep Dive into LLMs Like ChatGPT by Andrej Karpathy

#54

Post by 2-3-7-43-1807 »

dzogchen wrote: post #2

For a model to be ‘fully’ open source you need more than the model itself and a way to run it. You also need the data and the program that can be used to train it. See The Open Source AI Definition from OSI: https://opensource.org/ai

> you need more than the model itself and a way to run it. You also need the data and the program that can be used to train it.

the model reveals the architecture which is all you need to use/run/train it.

Re: TL;DR of Deep Dive into LLMs Like ChatGPT by Andrej Karpathy

#55

Post by 2-3-7-43-1807 »

est wrote: post #20

I have read many articles about LLMs, and understand how it works in general, but one thing always bothers me: why other models did't work as good as SOTA ones? What's the history and reason behind the current model architecture?

what do you refer to by "other models" not belonging to the "SOTA ones"?

Re: TL;DR of Deep Dive into LLMs Like ChatGPT by Andrej Karpathy

#56

Post by quantumspandex »

Andrej's video is great but the explanation on the RL part is a bit vague to me. How exactly do we train on the right answers? Do we collect the reasoning traces and train on them like supervised learning or do we compute some scores and use them as a loss function ? Isn't the reward then very sparse? What if LLMs can't generate any right answers cause the problems are too hard?

Also how can the training of LLMs be parallelized when updating parameters are sequential? Sure we can train on several samples simultaneously, but the parameter updates are with respect to the first step.

Re: TL;DR of Deep Dive into LLMs Like ChatGPT by Andrej Karpathy

#57

Post by FeepingCreature »

2-3-7-43-1807 wrote: post #53
albert_e wrote: post #12

OT: What is a good place to discuss the original video -- once it has dropped out of the HN front-page? I am going through the video myself -- roughly halfway through -- and have a fw things to bring up. Here they are now that we have a fresh opportunity to discuss: 1 - MATH and LLMs I am curious why many of the examples Andrej chose to pose to the LLM were "computational" questions -- for instance "what is 2+2" or s…

> for instance "what is 2+2" or some numerical puzzles that needed algebraic thinking there is only one algebraic approach to solving something like 2+2 and that is counting! 2+2 = (((0 + 1) + 1) + 1) + 1). but llms are infamously bad at counting. which is why 2+2 isn't an algebraic problem to an llm. it's pattern matching or linguistic reasoning token by token.

LLMs are bad at counting because nobody counts in text, we count in our heads which is not in the training material.

Re: TL;DR of Deep Dive into LLMs Like ChatGPT by Andrej Karpathy

#58

Post by mtkd »

quantumspandex wrote: post #56

Andrej's video is great but the explanation on the RL part is a bit vague to me. How exactly do we train on the right answers? Do we collect the reasoning traces and train on them like supervised learning or do we compute some scores and use them as a loss function ? Isn't the reward then very sparse? What if LLMs can't generate any right answers cause the problems are too hard? Also how can the training of LLMs be p…

Details on how DS used GRPO for RL rewards

https://medium.com/@sahin.samia/the-math-behind-deepseek-a-d...

Re: TL;DR of Deep Dive into LLMs Like ChatGPT by Andrej Karpathy

#59

Post by epr »

quantumspandex wrote: post #56

Andrej's video is great but the explanation on the RL part is a bit vague to me. How exactly do we train on the right answers? Do we collect the reasoning traces and train on them like supervised learning or do we compute some scores and use them as a loss function ? Isn't the reward then very sparse? What if LLMs can't generate any right answers cause the problems are too hard? Also how can the training of LLMs be p…

https://arxiv.org/abs/1707.06347

Re: TL;DR of Deep Dive into LLMs Like ChatGPT by Andrej Karpathy

#60

Post by fenomas »

quantumspandex wrote: post #56

Andrej's video is great but the explanation on the RL part is a bit vague to me. How exactly do we train on the right answers? Do we collect the reasoning traces and train on them like supervised learning or do we compute some scores and use them as a loss function ? Isn't the reward then very sparse? What if LLMs can't generate any right answers cause the problems are too hard? Also how can the training of LLMs be p…

As I understood that part, in RL for LLMs you take questions for which the model already sometimes emits correct answers, and then repeatedly infer while reinforcing the activations the model made during correct responses, which lets it evolve its own ways of more reliably reaching the right answer.

(Hence the analogy to training AlphaGo, wherein you take a model that sometimes wins games, and then play a bunch of games while reinforcing the cases where it won, so that it evolves its own ways of winning more often.)

Post reply on HN