11–20 of 103 posts
This appears to be a basically content-free slideshow with very shallow thinking...
(Or am I now like the Dwarfs in The Last Battle, seeing all things as mere simulacra?)
This appears to be a basically content-free slideshow with very shallow thinking...
To take a simple example: I grew up with computer games in the '80s where there were no 'physics engines' or frameworks for building games. As a result, each game was an expression of the author's personality somehow. Fast forward to the noughties, games bored me as they mostly looked and felt the same, or maybe felt like 3-5 different games all packaged differently.
Another example: going abroad on holiday in Europe (I'm from London) used to be a relatively wild, vibrant experience, filled with unexpected differences and challenges (not all positive). There were no McDonalds or Starbucks and the shops were filled with unfamiliar products and foods. Now everywhere in Europe feels the same when I visit, especially with smartphone in hand.
And films went from wildly different to one another to what now feels like 'arty' and 'CGI' being the two choices.
This article continues that into the realm of ideas, or idea production. Everywhere you go looks and feels familiar.
Or am I just getting old?
This is my biggest concern. Speaking as someone who recently had to read 60 AI generated reports (the whole issue of how much students are using AI is one discussion), it was genuinely soul-destroying reading the same phrases, seem sentence structures, same arguments over and over. Depressed me the whole of the next day.
There's a very interesting opportunity here for a deeper investigation.
- What does "regression to the mean" actually mean in practice when the LLM is conditioned on a possibly large amount of context?
- How does this perceived regression to the mean affect the result in different applications? When implementing code, it may show up as keeping it simple, hence easily understandable, "nonclever". When writing documentation, it may show up as simple language, short sentences, etc. supporting the intent of communicating with little friction to a broad audience. When brainstorming product ideas, it may show up as regurgitating old and boring ideas, but dressed in fancy language and affirmations that hide the shallowness of the content.
- What can be done to alter this behavior? Now that temperature doesn't seem to be a parameter anymore in new models, how can we steer creativity of the model?
- If the model's creativity is fundamentally limited, is there a way we can use it to support us in the expression of our creativity, leveraging the different strengths of humans and LLMs in a way that the result transcends the limits of either?
Unfortunately, I don't see the article doing that. And, while I know pointing out LLM-isms is often a cheap shot these days, I feel compelled to point out that this article is full of what I perceived as LLM-ism, quite ironic given the premise and the statement ("written off-distribution · on purpose").
E.g.
> Trained on the past, it answers in the past tense of thought. Not what is true. What is typical.
> We converge — not on what is right, but on what is average.
> Not the answer it was sure of — the one it would not stop correcting
This touches on something I've (and many others) have felt throughout my life, not just since the advent of LLMs. To take a simple example: I grew up with computer games in the '80s where there were no 'physics engines' or frameworks for building games. As a result, each game was an expression of the author's personality somehow. Fast forward to the noughties, games bored me as they mostly looked and felt the same, o…