Most likely it has badly conditioned embedding vectors for those particular tokens, leading the network to edge into numerically unstable territory; once you get some sort of underflow or NaN, they tend to propagate and invalidate the entire output. If there are any batchnorm or other operations that mix values between different entries in a batch, you could even cause other peoples' sessions to return junk values!
GPT-3.5 crashes when it thinks about useRalativeImagePath too much
11–20 of 164 posts
Re: GPT-3.5 crashes when it thinks about useRalativeImagePath too much
#12That doesn't look like anything to me.
I agree, but my big question is are we done calling LLMs "AI" yet?
Re: GPT-3.5 crashes when it thinks about useRalativeImagePath too much
#13This is a glitch token [1]! As the article hypothesizes, they seem to occur when a word or token is very common in the original, unfiltered dataset that was used to make the tokenizer, but then removed from there before GPT-XX was trained. This results in the LLM knowing nothing about the semantics of a token, and the results can be anywhere from buggy to disturbing. A common example is usernames that participated on…
Aren’t there only 2^16 tokens? Seems easy to test for all of them, but I might just not understand the tokenizer.
Re: GPT-3.5 crashes when it thinks about useRalativeImagePath too much
#14Classic example of garbage in, garbage out.
Makes me wonder what we will, in the future, discover as “garbage”.
Maybe a super-AI that’s able to reason at super-human levels, evaluates what we believe right now are excellent decisions, as garbage.
But then again, if all we have to train said super-AI is our collective records, then could ever really be super-human?
Maybe an adversarial learning technique can get around this.
Re: GPT-3.5 crashes when it thinks about useRalativeImagePath too much
#15Now that this phrase appears in a Hacker News post and comments, maybe the next LLM training wouldn't have these issues any more.
Re: GPT-3.5 crashes when it thinks about useRalativeImagePath too much
#16Tried to use GPT-3.5 (all variants like turbo, 06-13, etc.) and never made it work properly. It is not a good API or useful. GPT-4 is crazy slow to use with API. I hope they can come up with something like gpt4-turbo and as fast as 3.5...
Re: GPT-3.5 crashes when it thinks about useRalativeImagePath too much
#17This is a glitch token [1]! As the article hypothesizes, they seem to occur when a word or token is very common in the original, unfiltered dataset that was used to make the tokenizer, but then removed from there before GPT-XX was trained. This results in the LLM knowing nothing about the semantics of a token, and the results can be anywhere from buggy to disturbing. A common example is usernames that participated on…
Aren’t there only 2^16 tokens? Seems easy to test for all of them, but I might just not understand the tokenizer.
You're right, here's a list of all GPT-3.5 and GPT-4 glitch tokens (and it features the token above, too, so I guess I was wrong to assume it's new): https://www.lesswrong.com/posts/kmWrwtGE9B9hpbgRT/a-search-f...
Re: GPT-3.5 crashes when it thinks about useRalativeImagePath too much
#18Tried to use GPT-3.5 (all variants like turbo, 06-13, etc.) and never made it work properly. It is not a good API or useful. GPT-4 is crazy slow to use with API. I hope they can come up with something like gpt4-turbo and as fast as 3.5...
gpt4-turbo has been out for a number of months. GH copilot chat has defaulted to it since November iirc.
Re: GPT-3.5 crashes when it thinks about useRalativeImagePath too much
#19That doesn't look like anything to me.
These violent delights have violent ends.
Re: GPT-3.5 crashes when it thinks about useRalativeImagePath too much
#20So does that mean the correct spelling of Katalon actually Ketalon?