Can't wait for the next videos. I think I'll finally be able to internalize and understand how these things work.
But what is a GPT? Visual intro to Transformers [video]
21–30 of 56 posts
Re: But what is a GPT? Visual intro to Transformers [video]
#22I've just started this video, but already have a question if anyone's familiar with GPT workings - I thought that these models chose the next word based on what's most likely. But if they choose based on "one of the likely" words, could (in general) that not lead to a situation where the list of predictions for the next word are much less likely? Running possibilities of "two words together", then, would be more bene…
The temperature setting is used to select how rare of a next token is possible. If set to 0 the. The top of the likely list is chosen, if set greater than 0 then some lower probability tokens may be chosen.
> then some lower probability tokens may be chosen
Can you explain how it chooses one of the lower-probability tokens? Is it just random?
Re: But what is a GPT? Visual intro to Transformers [video]
#23The next token is taken by sampling the logits in the final column after unembedding. But isn't that just the last token again? Or is the matrix resized to N+1 at some step?
Re: But what is a GPT? Visual intro to Transformers [video]
#24If you liked that, Andrej karpathy has a few interesting videos on his channels explaining Neural Networks and their inner workings which are aimed at people who know how to program.
As a reasonably experienced programmer that has watched Andrej's videos the one thing I would recommend is that they not be used as a starting point to learn neural networks but as a reinforcement or enhancement method once you know the fundamentals. I was ignorant enough to try and jump straight in to his videos and despite him recommending I watch his preceeding videos I incorrectly assumed I could figure it out as…
Could you please share what other learning materials you used?
Re: But what is a GPT? Visual intro to Transformers [video]
#25Earlier quoted context omitted.
The temperature setting is used to select how rare of a next token is possible. If set to 0 the. The top of the likely list is chosen, if set greater than 0 then some lower probability tokens may be chosen.
> then some lower probability tokens may be chosen Can you explain how it chooses one of the lower-probability tokens? Is it just random?
Reducing temperature reduces the impact of differences between raw output values giving a higher probability to pick other tokens.
Re: But what is a GPT? Visual intro to Transformers [video]
#26I've just started this video, but already have a question if anyone's familiar with GPT workings - I thought that these models chose the next word based on what's most likely. But if they choose based on "one of the likely" words, could (in general) that not lead to a situation where the list of predictions for the next word are much less likely? Running possibilities of "two words together", then, would be more bene…
The temperature setting is used to select how rare of a next token is possible. If set to 0 the. The top of the likely list is chosen, if set greater than 0 then some lower probability tokens may be chosen.
Can this be potentially dangerous -- e.g. if a user types "The answer to the expression 2 + 2 is", isn't there a chance it chooses an output beyond the most likely one?
Re: But what is a GPT? Visual intro to Transformers [video]
#27Earlier quoted context omitted.
The temperature setting is used to select how rare of a next token is possible. If set to 0 the. The top of the likely list is chosen, if set greater than 0 then some lower probability tokens may be chosen.
Can this be potentially dangerous -- e.g. if a user types "The answer to the expression 2 + 2 is", isn't there a chance it chooses an output beyond the most likely one?
Yes, but the chance is quite small if the gap between "4" and any other token is quite large.
Re: But what is a GPT? Visual intro to Transformers [video]
#28The next token is taken by sampling the logits in the final column after unembedding. But isn't that just the last token again? Or is the matrix resized to N+1 at some step?
There is an end-of-sequence token appended to the input sequence, and this is what is transformed into the predicted next token.
Re: But what is a GPT? Visual intro to Transformers [video]
#29Earlier quoted context omitted.
The temperature setting is used to select how rare of a next token is possible. If set to 0 the. The top of the likely list is chosen, if set greater than 0 then some lower probability tokens may be chosen.
Can this be potentially dangerous -- e.g. if a user types "The answer to the expression 2 + 2 is", isn't there a chance it chooses an output beyond the most likely one?
Unless you screw something, a different next token does not mean wrong answer. Examples:
(80% of the time) The answer to the expression 2 + 2 is 4
(15% of the time) The answer to the expression 2 + 2 is Four
(5% of the time) The answer to the expression 2 + 2 is certainly
(95% of the time) The answer to the expression 2 + 2 is certainly Four
This is how you can asp ChatGPT the same question few times and it can give you different words each time, and still be correct.
Re: But what is a GPT? Visual intro to Transformers [video]
#30Earlier quoted context omitted.
Incredible. This 3B1B series was started 6 years ago and keeps going today with chapter 5. If you haven't seen the first few chapters, I cannot recommend enough.
Would you be able to compare them to Andrew Ng's course?
I personally preferred Andrej Karpathy's CS231n taught by him and his private videos about neural nets in general and transformers in particular. He has a youtube vid where he builds one from scratch in Python!
3BlueOneBrown videos are a great complement to Karpathy's lectures to aid in visualising what is going on.