Earlier quoted context omitted.
You need filter assisted decoding to make that possible in all cases (as in, working for both models all the time provably) I'm surprised it worked in chatGPT this time, but it won't work all the time due to lack of lexical, semantic, or phonetic info in the tokenizer. https://paperswithcode.com/paper/most-language-models-can-be...
It works in GPT-4 (but not GPT-3.5) even without a special tokenizer: Some problems simply solve themselves with increasing scale. But Bard uses a smaller LaMDA model to reduce inference cost. Quote [1] from Sundar Pichai in February: > We’re releasing it initially with our lightweight model version of LaMDA. This much smaller model requires significantly less computing power, enabling us to scale to more users, allo…
I'm impressed that it is working more often than not, but we need provable guarantees.