what's wild is they accidentally solved it — pretraining IS unsupervised learning at scale, RLHF IS reinforcement learning. they just didnt know the recipe yet
pretraining isn't unsupervised, it is self-supervised - meaning it is moderately more scale limited.
What would unsupervised mean, would unsupervised be something like alphago playing against itself trillions of times?
Whereas self-supervised, allows learning without explicit annotation of data ; but it doesn't matter if the models already trained on the entire Internet, and it's not like a game where it can come up with effectively new training data for itself?