Understanding GPT tokenizers

Sure, the concept has been explored; for example see the classic 2015 Karpathy's http://karpathy.github.io/2015/05/21/rnn-effectiveness/ as a cool description of a character-level model.

IIRC the early papers on subword tokenization also sometimes included explicit comparisons with character-level models, but people don't do it nowadays because there's a clear consensus on the expected outcome - yes, it works, but it's simply worse.

Technically it's the exact outcome that you get if you put in a vocabulary size of 256 (and do tokenization on byte-level, not unicode), so it's just an extreme case of vocabulary size choice, and there's enough research on how vocabulary size affects stuff to assume that 256 is not an optimal size.

You can do it for exploring capabilities though - see "Bytes is all you need" https://news.ycombinator.com/item?id=36176756 discussion on trying to abstract away complex file formats by just passing the bytes of the file to the neural network directly - again, it obviously works worse, but it kind of works.

I can see the theoretical advantages of such a concept, but I think a key limitation is that we don't have appropriate amounts of data with accurate phonetic representation.

The potential advantage of using a phonetic representation is that it can have different relevant information than written spelling does. However, if you take the written spelling and pass it through some rules that transform it to what the phonetic representation might be... that transformation can only destroy information, not add it; you'd just be better off using the source data directly.

Now if at some point we get to a place where most of the training data is audio (i.e. the quantity of spoken words in available audio data becomes larger than current written data on internet and in libraries), then phonetic representation would make all sense, being closer to the source data.

But if we're talking about purely tokenization - I think your suggestion is effectively halfway towards morphologically based tokenization, splitting into morphemes (which tend to map to semantics), and that is getting explored. The problem is, for an apples-to-apples comparison you need equally sized models and changes to tokenization require a complete retraining of the model; so doing a comparison on GPT-3 or GPT-4 scale is very expensive (too expensive for "would be interesting" to justify it), and measuring the effect on small models won't necessarily be very indicative of how it will affect large models.

Read that thread -- the summaries and glossary it gives are very helpful.

My brother needed to take a certification test for his (non-technical) job, and he had a bunch of dead-trees to study...

So I asked chatGPT to summarize each section of the study material (a national test for a trade) -- which it did

I then asked it for smaple questions which would reflect the test for each section, and it did.

> However, once they've started writing rhyming poetry it's hard to get them to stop rhyming. They seem to have formed a strong association between rhyming and poetry. I've also been unable to get them to obey a specific rhyming scheme like ABBAB.

Correct and commonly observed (eg. https://arxiv.org/abs/2305.11064 ). (At least, for GPT models. I don't know as much about the Anthropic models as I should, although I understand they do still use a BPE tokenization, unfortunately.) My theory is that it is a surprising interaction of BPEs with RLHF: https://www.lesswrong.com/posts/t9svvNPNmFf5Qa3TA/mysteries-...

As much as they look like they can, they can't rhyme because of BPEs still. What they have done in lieu of genuine phonetic understanding is, more or less, memorized a ton of rhyme-pairs: they only have a vast patchwork of half-understood phonetics discerned dimly through the lossy compression of BPEs and memorized pairs. If you don't force them out of the memorized space and let them write without interruption, they look like they understand, but they still don't.

Then RLHF punishes them for any incorrect poetry, so they never leave the memorized space on their own because that's the only way to guarantee correct rhyming poetry. And since there is no way for it to tell the difference between 'rhymes but I don't know that it rhymes because BPEs' and 'deliberately nonrhyming poetry', much less what the difference is between 'ABBAB' and 'AABBAA', it just always does rhyming quatrains etc. Why take the risk?

Also applies to jokes and joke explanations: https://arxiv.org/abs/2306.04563 It can't understand properly what is a joke or not, because it's blind to what makes a vast number of jokes work, so it just memorizes a few safe jokes and assumes anything presented to it as a joke must be one of the countless jokes that it can't understand & makes up its best guess.