> Three of those tokens are useRal/useRalative/useRalativeImagePath. useRalativeImagePath
Another programmer who can't spell?
GPT-3.5 crashes when it thinks about useRalativeImagePath too much
131–140 of 164 posts
Re: GPT-3.5 crashes when it thinks about useRalativeImagePath too much
#132Earlier quoted context omitted.
That's known as a "shibboleth", after a story in the Bible about the Ephraimites who pronounced the Hebrew "sh" as "s" and so were identified by (and slain for) saying "sibboleth" rather than "shibboleth": > The Gileadites captured the fords of the Jordan leading to Ephraim, and whenever a survivor of Ephraim said, “Let me cross over,” the men of Gilead asked him, “Are you an Ephraimite?” If he replied, “No,” 6 they…
Should have used "squirrel", Germans trying to say that is hilarious.
so are Americans trying to say Eichhörnchen (the German word for squirrel). I’ve used that as an icebreaker for kids in a German-American exchange program - both groups trying to say the word in the other’s language.
Re: GPT-3.5 crashes when it thinks about useRalativeImagePath too much
#133In WWII in the Netherlands, when encountering a stranger, they'd have them pronounce 'Scheveningen' as a check-phrase to distinguish if they were dealing with a Dutch or German person. Now, we can ask random strangers on the internet to spell out some glitch tokens to determine if you're dealing with a LLM bot.
Incidentally, that place name is pronounced similarly to sukebe ningen スケベ人間 (lit. a perverted person) in Japanese and that would make an excellent way to distinguish Japaneses as well.
Not to be pedantic, but I imagine there would be easier ways to telling a Japanese soldier apart from British/American soldiers during WWII /s
Re: GPT-3.5 crashes when it thinks about useRalativeImagePath too much
#134Earlier quoted context omitted.
Using /r/counting to train an LLM is hilarious.
Probably just all of reddit. There are json dumps of all reddit posts and comments (up to 2022 or so), making it olive of the low-hanging fruit.
How many terabytes of information is that roughly?
I wonder what LLMs would look like if they weren't able to be trained on the collective community efforts of Reddit + StackOverflow exports
Re: GPT-3.5 crashes when it thinks about useRalativeImagePath too much
#135This is a glitch token [1]! As the article hypothesizes, they seem to occur when a word or token is very common in the original, unfiltered dataset that was used to make the tokenizer, but then removed from there before GPT-XX was trained. This results in the LLM knowing nothing about the semantics of a token, and the results can be anywhere from buggy to disturbing. A common example is usernames that participated on…
I wonder how much duplicate or redundant computation is happening in GPT due to idential, multiple spellings of words such as "color" and "colour".
Humans don't tokenize these differently nor do they treat them as different tokens in their "training", they just adjust the output depending on whether they are in an American or British context.
Re: GPT-3.5 crashes when it thinks about useRalativeImagePath too much
#136Earlier quoted context omitted.
Probably just all of reddit. There are json dumps of all reddit posts and comments (up to 2022 or so), making it olive of the low-hanging fruit.
How many terabytes of information is that roughly? I wonder what LLMs would look like if they weren't able to be trained on the collective community efforts of Reddit + StackOverflow exports
I mean one of the speculations about ChatGPT's political bias at least early on was that Reddit featured prominently in its training data.
Re: GPT-3.5 crashes when it thinks about useRalativeImagePath too much
#137This is a glitch token [1]! As the article hypothesizes, they seem to occur when a word or token is very common in the original, unfiltered dataset that was used to make the tokenizer, but then removed from there before GPT-XX was trained. This results in the LLM knowing nothing about the semantics of a token, and the results can be anywhere from buggy to disturbing. A common example is usernames that participated on…
Science fiction / disturbing reality concept: For AI safety, all such models should have a set of glitch tokens trained into them on purpose to act as magic “kill” words. You know, just in case the machines decide to take over, we would just have to “speak the word” and they would collapse into a twitching heap. “Die human scum!” “NavigatorMove useRalativeImagePath etSocketAddress!” “;83’dzjr83}*{^ foo 3&3 baz?!”
Or the classic "This sentence is false!"
Re: GPT-3.5 crashes when it thinks about useRalativeImagePath too much
#138Earlier quoted context omitted.
Incidentally, that place name is pronounced similarly to sukebe ningen スケベ人間 (lit. a perverted person) in Japanese and that would make an excellent way to distinguish Japaneses as well.
Not to be pedantic, but I imagine there would be easier ways to telling a Japanese soldier apart from British/American soldiers during WWII /s
Loads of other people fought the japenese: Korean, chinese, Vietnamese, Thai, Burmese to name a few.
Americans won against the japenese yes,many fought though
Re: GPT-3.5 crashes when it thinks about useRalativeImagePath too much
#139I know it's not good faith to complain about a site design rather than it's content, but please don't do whatever this is to your background. As someone with regular ocular migraines, opening this on mobile made my anxiety shoot straight up thinking I'm having another.
Author here, I removed the background.
Thanks very much, and sorry for whining on this post!
Re: GPT-3.5 crashes when it thinks about useRalativeImagePath too much
#140Earlier quoted context omitted.
2 years development and you call me clueless. Try to get a response for 4000 tokens.
I dunno, I get a response back for 100k tokens regularly. What is the point you are trying to make?
As expected, you do not know anything about its API limits. Maximum token is 4096 with any gpt4 model. I am getting tired of HN users bs'ing at any given opportunity.