Regression to the Mean: on LLMs and the quiet death of the new

First of all, neural nets do nit return averages per se. They construct space between the points and extrapolate outside of the points. So even if a point was not in their training data, they will be ok, in many situations, to acknowledge it.

Or in other words - LLMs don’t average. They construct world models. A novel thing that fits their world model will be accepted no prob. A thing that doesn’t may still be accepted but with challenges.

The same is true though for humans, including scientists. There is a saying that science moves one grave at a time - because often prev gen of scientists needs to die off for a new idea to take root.

Or in yet other words - even if llms produced averages, an average of a discontinuous set can lie outside of that set. And the set of all human ideas is very much discontinuous.

Both games and movies are predictable in the sense that we know what to expect, and they have been largely standardized. Games have common keybinding schemes, as well as user experience mechanics: how jumping feels, when we expect to autosave, what the UI/minimap symbols mean, etc. When it comes to movies, I find myself no longer turning away from the screen before gruesome scenes, because I expect in advance that they won't show it, depending on the mood of the movie. I also find that you can often predict which dialogue lines were meant as foreshadowing for a plot twist coming later. This standardization is intentional in the sense that people are more likely to consume something they are familiar with, and more likely to enjoy it if they can passively engage with it.

It's common nowadays to pay $20 for a game, play it for a few hours, and forget about it. Or, turn on a random Netflix show on the TV to pass time in the evening. Quite likely that a month later you won't reminisce about either of these experiences, but you probably didn't have high expectations either way. I think 'consuming' a travel trip is similar in the sense that it has very familiar tropes no matter where you go, but more implicitly resulting from market forces rather than intentional design from a creator.

https://mapwriting.substack.com/p/living-subscription-free-i...

That's a common meme but it's the opposite of true. Everything big models, not just transformers, mathematically do is extrapolation in the feature space, almost never interpolation. They're perfectly able of combining the ideas, although of course this ability declines once they're off the distribution, just like in humans. The model is creative and its output is transformative, however it only makes sense if you define creativity in a pure manner, based on novelty for itself.

However most people use entirely different definitions of creativity, something like "surprise me in a way that still makes sense to me". This includes "me", a side observer, and depends on what the side observer considers novel. Hence the main reason for the lack of "creativity" in big models is not some hand-waved "regression to the mean" or "interpolation", but the fact that they're still insufficiently intelligent compared to a human. Current models simply don't have enough fidelity to understand humans and think as deep, that's why humans think their output isn't sufficiently novel for them! Think about this, insanely smart aliens would probably dismiss humans as non-creative too.

The contributing factor is also the lack of semantic diversity in current models. Also known as mode collapse, but the name is a bit of misnomer and describes a technicality, not the resulting phenomenon. This indirectly affects creativity as it's usually understood, because the models generate and repeat the same thing in response to the same thing, which is the opposite of novel in layman's definition. That's part of where the slop comes from. Mode collapse has many causes, e.g. post-training with current algorithms. It's likely fixable but AI shops show little to no interest in studying and fixing it.