Can anyone comment on how the limits on GPT-X’s token space translate to limits on its vocabulary (with corresponding limits on understanding input and generating output)?
For example, is GPT-4’s list of ~100k tokens sufficient to understand and generate every non-obsolete word in the English language (per, say, a standard dictionary)? Or even every word in the training data?
If not, do we have examples of ordinary words that it is impossible for GPT-4 ever to understand or generate? What happens when it encounters those words and is unable to tokenize them; are they simply ignored (eg omitted from the input vector, or set to 0 or some sort of null token)?
Can anyone comment on how the limits on GPT-X’s token space translate to limits on its vocabulary (with corresponding limits on understanding input and generating output)? For example, is GPT-4’s list of ~100k tokens sufficient to understand and generate every non-obsolete word in the English language (per, say, a standard dictionary)? Or even every word in the training data? If not, do we have examples of ordinary w…
My current understanding is that the lack of a token for a specific word does nothing to prevent that word from being "understood" or produced in output - GPT-4 is very capable of consuming and producing text in languages such as Spanish despite most Spanish words not corresponding to a single token.
The tokens are an integer. The first layer of the model is an 'embedding', which is essentially a giant lookup table. So if a string gets tokenized to Token #3, that means get the vector in row 3 of the embedding table. (Those vectors are learned during model training.) More completely, you can think of the integers as being implicitly a one-hot vector encoding. So say you have a vocab size of 20,000 and you want Tok…
Kind of refreshing to see this perspective on lookup vs matrix multiplication, specially with the bias towards the latter as more natural. Is there some reference table somewhere mapping more code idioms like this to equivalent nn representations?
Hmm not that I know of, but that would be neat! A lot of the frameworks and model code treat this sort of thing as ‘implementation details’. Which is disappointing because I think it adds perspective and intuition.
One other example would be how multi-head attention is implemented with a single matrix. You don’t actually create matrices for each of the N ‘heads’ separately. It’s a logical distinction
Can anyone comment on how the limits on GPT-X’s token space translate to limits on its vocabulary (with corresponding limits on understanding input and generating output)? For example, is GPT-4’s list of ~100k tokens sufficient to understand and generate every non-obsolete word in the English language (per, say, a standard dictionary)? Or even every word in the training data? If not, do we have examples of ordinary w…
My current understanding is that the lack of a token for a specific word does nothing to prevent that word from being "understood" or produced in output - GPT-4 is very capable of consuming and producing text in languages such as Spanish despite most Spanish words not corresponding to a single token.
For Russian text, it degrades to, basically, 1 character = 1 token, due to the tokenization issues discussed in the article, yet it produces absolutely coherent text, almost same as in English. In my tests, its Russian output is worse than English output, though, something like 80% quality I'd say. I'm not an LLM expert but I have a theory that, being mostly trained on English text, its thought processes actually happen in English (the part of the model which was trained on English text) and for Russian, it's able to map English to Russian and back thanks to its language translation ability, because I've noticed sometimes it produces slightly awkward sentences whose word choice makes sense in English (calques?) and not as much in Russian.
Can anyone comment on how the limits on GPT-X’s token space translate to limits on its vocabulary (with corresponding limits on understanding input and generating output)? For example, is GPT-4’s list of ~100k tokens sufficient to understand and generate every non-obsolete word in the English language (per, say, a standard dictionary)? Or even every word in the training data? If not, do we have examples of ordinary w…
IIRC from poking around in the LLaMA internals (I assume ChatGPT is the same since it’s the obvious way to handle this): the token list has a complete set of tokens of length 1. This means that in the degenerate case where the tokenizer can’t compose the text out of any other tokens it’ll still be processable, just as a collection of single-character tokens that the language model presumably has vaguer associations for. (Which I imagine doesn’t actually affect things significantly; if you added more tokens for less-frequently-seen strings, it still wouldn’t have much of an idea what to do with them.)
Tokens are the primitives that most LLMs (and broadly a lot of NLP) works with. While, you and I would expect whole-words to be tokens, many tokens are shorter - 3 to 4 characters - and don't always match the sentence structure you and I expect. This can create some interesting challenges and unexpected behavior. It also makes certain things, like vectorization, a challenge since tokens may not map 1:1 with the words…
Your answer explains what tokenizers are, which isn't what I asked. You also told me something interesting about tokenizers, which is also not what I asked. Can you tell me anything NOT about tokenized? This is my point.
You asked why there was a focus on discussing tokenizers, and got an answer explaining why tokenizers are something people would want to discuss.
Tokens are the primitives that most LLMs (and broadly a lot of NLP) works with. While, you and I would expect whole-words to be tokens, many tokens are shorter - 3 to 4 characters - and don't always match the sentence structure you and I expect. This can create some interesting challenges and unexpected behavior. It also makes certain things, like vectorization, a challenge since tokens may not map 1:1 with the words…
> While, you and I would expect whole-words to be tokens, many tokens are shorter - 3 to 4 characters - and don't always match the sentence structure you and I expect. There is a phenomenon called Broca's Aphasia which is, essentially, the inability to connect words into sentences. This mostly prevents the patient from communicating via language. But patients with this condition can reveal quite a bit about the struc…
You can observe (what I assume is) the same tokenization phenomenon in people who are struggling to speak (for example because they’re distracted by something or not native speakers): stock fragments will come out all at once, and less common words will get split, usually on affixes or at the join point of compound words.
In the article on your blog, you wrote: "GPT-3 rhymes reasonably well and often when appropriate, but the improvement is much smaller on rhyming than it is on pretty much everything else. Apparently it is easier for GPT-3 to learn things like arithmetic and spreadsheets than it is to learn how to rhyme." I've experimented extensively with Claude, and a bit with Claude+, ChatGPT (GPT 3.5) and GPT4 on poe.com, and I've…
> However, once they've started writing rhyming poetry it's hard to get them to stop rhyming. They seem to have formed a strong association between rhyming and poetry. I've also been unable to get them to obey a specific rhyming scheme like ABBAB. Correct and commonly observed (eg. https://arxiv.org/abs/2305.11064 ). (At least, for GPT models. I don't know as much about the Anthropic models as I should, although I un…
I wonder if having access to characters actually helps rhyming in English all that much, as English rules of pronunciation are essentially rote-learned anyway. If it were not rote-learning, then it might make different mistakes, for example expecting two words to rhyme because they end with the same suffix.
Perhaps it would be more effective to ask it to produce poems in the format: English0 IPA0 English1 IPA1, where each line is produced in both semantic and phonetic representations. This would give it the context necessary to “see” the rhymes without having to mess around with the tokenization.
I frequently see people surprised about the kinds of "simple" mistakes LLMs make when given tasks that involve letters, syllables, word lengths, rhyming, etc. They don't do well at character oriented tasks which seem trivial to us, like "write me a sentence with only 5 letter words". All of these problems stem from the fact that LLMs don't "see" the actual letters/characters in the text they are consuming and produci…
Yup! I wrote a whole paper and constrained text generation studio about this which has gotten some positive attention:
I really really wish someone would try tokenizing off of a phonetic representation rather than textual one. I think it would be interesting to compare the output
What do you mean by phonetic representation? Sound files?