Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

But what is a GPT? Visual intro to Transformers [video]

youtube.com

Post reply on HN
Print view View original

31–40 of 56 posts

Re: But what is a GPT? Visual intro to Transformers [video]

#31

Post by chessgecko »

user_7832 wrote: post #5

I've just started this video, but already have a question if anyone's familiar with GPT workings - I thought that these models chose the next word based on what's most likely. But if they choose based on "one of the likely" words, could (in general) that not lead to a situation where the list of predictions for the next word are much less likely? Running possibilities of "two words together", then, would be more bene…

There’s some fancier stuff too like techniques that take into account where recent tokens were drawn from in the distribution and update either the top_p or the temperature so that sequences of tokens have a minimum unlikeliness. Beam search is less common with really large models because the computation is really expensive.

Re: But what is a GPT? Visual intro to Transformers [video]

#32

Post by weitendorf »

Habgdnv wrote: post #29
DrawTR wrote: post #26

Earlier quoted context omitted.

Can this be potentially dangerous -- e.g. if a user types "The answer to the expression 2 + 2 is", isn't there a chance it chooses an output beyond the most likely one?

Unless you screw something, a different next token does not mean wrong answer. Examples: (80% of the time) The answer to the expression 2 + 2 is 4 (15% of the time) The answer to the expression 2 + 2 is Four (5% of the time) The answer to the expression 2 + 2 is certainly (95% of the time) The answer to the expression 2 + 2 is certainly Four This is how you can asp ChatGPT the same question few times and it can give…

That assumes that the model is assigning vanishingly small weights to truly incorrect answers, which doesn't necessarily hold up in practice. So I think "Unless you screw something" is doing a lot of work there

I think a more correct explanation would be that increasing temperature doesn't necessarily increase the probability of a truly incorrect answer proportionately to the temperature increase (because the same correct answer could be represented by many different sequences of tokens), but if the model assigns a non-zero value to any incorrect output after applying softmax (which it most likely does), increasing the temperature does increase the probability of that incorrect output being returned.

Re: But what is a GPT? Visual intro to Transformers [video]

#33

Post by weitendorf »

DrawTR wrote: post #26
doctoboggan wrote: post #7

Earlier quoted context omitted.

The temperature setting is used to select how rare of a next token is possible. If set to 0 the. The top of the likely list is chosen, if set greater than 0 then some lower probability tokens may be chosen.

Can this be potentially dangerous -- e.g. if a user types "The answer to the expression 2 + 2 is", isn't there a chance it chooses an output beyond the most likely one?

Yes, although it's also possible that the most likely token is incorrect and perhaps the next 4 most likely tokens would lead to a correct answer.

For example if you ask a model what is 0^0, the highest probability output may be "1", which is incorrect. The next most probable outputs may be words like "although", "because", "Due to", "unfortunately", etc. as the model prepares to explain to the user that the value of the expression is undefined; because there are many more ways to express and explain the undefined answer than there are to express a naively incorrect answer, the correct answer is split across more tokens so that even if eg the softmax value of "1" is 0.1 and across "although"+"because"+"due to"+"unfortunately">0.3, at temperature of 0, "1" gets chosen. At slightly higher temperatures, sampling across all outputs would increase the probability of a correct answer.

So it's true that increasing the temperature increases the probability that the model outputs tokens other than the single-most-likely token, but that might be what you want. Temperature purely controls the distribution of tokens, not "answers".

Re: But what is a GPT? Visual intro to Transformers [video]

#34

Post by x-complexity »

DrawTR wrote: post #26
doctoboggan wrote: post #7

Earlier quoted context omitted.

The temperature setting is used to select how rare of a next token is possible. If set to 0 the. The top of the likely list is chosen, if set greater than 0 then some lower probability tokens may be chosen.

Can this be potentially dangerous -- e.g. if a user types "The answer to the expression 2 + 2 is", isn't there a chance it chooses an output beyond the most likely one?

> Can this be potentially dangerous -- e.g. if a user types "The answer to the expression 2 + 2 is", isn't there a chance it chooses an output beyond the most likely one?

This is where the semi-ambiguity of the human languages helps a lot with.

There are multiple ways to answer with "4" that are acceptable, meaning that it just needs to be close enough to the desired outcome to work. This means that there isn't a single point that needs to be precisely aimed at, but a broader plot of space that's relatively easier to hit.

The hefty tolerances, redundancies, & general lossiness of the human language act as a metaphorical gravity well to drag LLMs to the most probable answer.

Re: But what is a GPT? Visual intro to Transformers [video]

#35

Post by acchow »

acchow wrote: post #25
davekeck wrote: post #22

Earlier quoted context omitted.

> then some lower probability tokens may be chosen Can you explain how it chooses one of the lower-probability tokens? Is it just random?

Reducing temperature reduces the impact of differences between raw output values giving a higher probability to pick other tokens.

Oops backwards. Increasing temperature...

Re: But what is a GPT? Visual intro to Transformers [video]

#36

Post by Newlaptop »

weitendorf wrote: post #33
DrawTR wrote: post #26

Earlier quoted context omitted.

Can this be potentially dangerous -- e.g. if a user types "The answer to the expression 2 + 2 is", isn't there a chance it chooses an output beyond the most likely one?

Yes, although it's also possible that the most likely token is incorrect and perhaps the next 4 most likely tokens would lead to a correct answer. For example if you ask a model what is 0^0, the highest probability output may be "1", which is incorrect. The next most probable outputs may be words like "although", "because", "Due to", "unfortunately", etc. as the model prepares to explain to the user that the value of…

Not sure if you were making a joke, but 0^0 is often defined as 1.

https://en.wikipedia.org/wiki/Zero_to_the_power_of_zero

Re: But what is a GPT? Visual intro to Transformers [video]

#37

Post by qeternity »

mvsin wrote: post #6
user_7832 wrote: post #5

I've just started this video, but already have a question if anyone's familiar with GPT workings - I thought that these models chose the next word based on what's most likely. But if they choose based on "one of the likely" words, could (in general) that not lead to a situation where the list of predictions for the next word are much less likely? Running possibilities of "two words together", then, would be more bene…

Something like this does exist, production systems rarely use greedy search but have more holistic search algorithms. An example is Beam Search: https://www.width.ai/post/what-is-beam-search Essentially we keep a window of probabilities of predicted tokens to improve the final quality of output.

> production systems rarely use greedy search

I have no idea why you say this. Most of our pipelines will run greedy, for reproducibility.

Maybe we turn the temp up if we are returning conversational text back to a user.

Re: But what is a GPT? Visual intro to Transformers [video]

#39

Post by ahzhou »

user_7832 wrote: post #5

I've just started this video, but already have a question if anyone's familiar with GPT workings - I thought that these models chose the next word based on what's most likely. But if they choose based on "one of the likely" words, could (in general) that not lead to a situation where the list of predictions for the next word are much less likely? Running possibilities of "two words together", then, would be more bene…

Yes, this is a fundamental weakness with LLMs. Unfortunately this is likely unsolvable because the search space is exponential. Techniques like beam search help, but can only introduce a constant scaling factor.

That said, LLM reach their current performance despite this limitation.

Re: But what is a GPT? Visual intro to Transformers [video]

#40

Post by weitendorf »

Newlaptop wrote: post #36
weitendorf wrote: post #33

Earlier quoted context omitted.

Yes, although it's also possible that the most likely token is incorrect and perhaps the next 4 most likely tokens would lead to a correct answer. For example if you ask a model what is 0^0, the highest probability output may be "1", which is incorrect. The next most probable outputs may be words like "although", "because", "Due to", "unfortunately", etc. as the model prepares to explain to the user that the value of…

Not sure if you were making a joke, but 0^0 is often defined as 1. https://en.wikipedia.org/wiki/Zero_to_the_power_of_zero

I honestly had forgot that, if I ever knew it. But I think the point stands that in many contexts you'd rather have the nuances of this kind of thing explained to you - able to represented by many different sequences of tokens, each individually being low probability - instead of simply taking the single-highest probability token "1".
Post reply on HN