Earlier quoted context omitted.
The shapes of clouds and positions of stars are essentially random, and yet humans derive meaning from both. I agree you could have gotten the same results via /dev/random, or probably by increasing the temperature on the model, but I suspect doing one of those things is important.
The LLM cannot derive meaning in a human sense. The shapes of clouds and positions of stars aren't completely random; there is useful information in them, to varying degrees (e.g. some clouds do look like, say, a rabbit, enough that a majority of people will agree). The mechanism at play here with the LLM is completely different; the connection between two dog-inputs and the resulting game barely exists, if at all. M…
It's not that the input contained useful information—obviously it does not—it's that it's causing the output to be more random, and thus more "creative".
Without the gibberish, "generate a random game" would likely repeatedly surface high-probability concepts—platformers, space shooters, tower defense—whatever sits near the top of the model's prior distribution for "game." The gibberish causes the model to land on concepts like "frog" that it would almost never reach otherwise.