"humanity discovered an algorithm that could really, truly learn any distribution of data (or really, the underlying “rules” that produce any distribution of data)..." This statement is manifestly untrue. Neural networks are useful, many hidden layers is useful, all of these architectures are useful, but the idea that they can learn anything, is based less on empirical results and more on what Sam Altman needs to con…
Six months ago, he probably could have gotten away with saying this and there would have likely have been enough people who were still impressed enough with the trajectory of LLMs to back him on it. But these days, most of us have encountered the all-too-common failure mode where the LLM shows its hand, that it doesn't truly understand anything, and that it's just _very very good_ at prediction. Each new generation gets even better at that prediction, but still hits its weird stumbling points, because its still the same algorithm, and that algorithm cannot do what he is ascribing to it.
These are the words of a man who has an incredible amount of money sunk into something and as such, is having a really hard time taking an honest accounting of it.