I think there is a rich seam of work for computers to help humans spot when there are anomalies in the hypotheses humans hold that can help humans form the corrections to hypothesis that fit our aesthetic sensibilities
Rich Sutton on AI creativity and discovery
71–80 of 141 posts
Re: Rich Sutton on AI creativity and discovery
#72The most successful applications like coding are not the result of pure LLM/generative modeling. They come from closing the loop with an agentic harness. The generate-test-selectively refine loop is the core modality of scientific work. An LLM + RL with Verifiable Rewards + feedback from compiler/terminal runs mimics this process to a great extend. This is Fisher/Box feedback loop ( https://www-sop.inria.fr/members/I…
Completely agree on the importance of the harness. The problem I see is the same problem Evolutionary Algorithms had: you can generate potential solutions until you run out of cash, but you still need to evalulate those solutions. You need a fitness function, and that means you need to at least know the general shape of the solution. If anyone knows of any work towards more open-ended fitness functions, I'd love to r…
Just some speculation, but, I think humans have on the one hand a lot of degrees of freedom in behaviors and thoughts they can do, but at the same time all that freedom is reigned in by our biological needs, like preserving the integrity of our body, but also preserve the integrity of our minds. But this extends further to preserving our surroundings (for our safety, a changing environment brings uncertainty), but also of people we care about and even entire societies that we have. And preserving our future selves through prediction of future environments.
So all that is to say, I'm not sure it is even theoretically possible to create a single algorithm to do open ended search and evaluation. Biology has billions of years of evolution and accumulation, whereas a simple algorithm in a computer, even if smart and connected to the real world, has no such accumulation.
I think humans hit the perfect sweet spot where we have the simplicity of the self preservation instinct, but we have the complexity of the cortex and lots of degrees of freedom because of it, plus on top of that we have a lot of accumulated degrees of freedom in the society and technology and knowledge that have we, which has been built up for thousands of years, all of which we can't just create an algorithm to encapsulate without going through the actual evolution.
And just to make it explicit - a large percentage of what humans think derives from an instinct to preserve the self, the mind, the future and the environment, even if it is very abstract at times. Not absolutely all, but I think a good chunk. And the complexity and degrees of freedom comes from that we have so many neurons in the brain, and a complex body with hands and whatever else that allows a lot of behaviors, as well as a complex environment that is constantly challenging us.
Re: Rich Sutton on AI creativity and discovery
#73The most successful applications like coding are not the result of pure LLM/generative modeling. They come from closing the loop with an agentic harness. The generate-test-selectively refine loop is the core modality of scientific work. An LLM + RL with Verifiable Rewards + feedback from compiler/terminal runs mimics this process to a great extend. This is Fisher/Box feedback loop ( https://www-sop.inria.fr/members/I…
Most importantly, the reinforcement loop is used during training. I don't agree with Sutton's original hypothesis, but it holds even less after reinforcement learning.
Re: Rich Sutton on AI creativity and discovery
#74Creativity is personal taste and the taste which captures the most is mainstream.
I tell jokes and the group of friends get them, for family they don't get them anymore.
I do not like 'basic jokes' and despite that, german television is full of it.
Most things in our world are more of a challange of finding the answers and not 'creating' the answers.
Math: the answer is already there, you only need to find it an fast space of posibilities. This is perfect for LLMs.
Creativity: a LLM can iterate over things a lot faster than a person. So we can iterate over this space too. We can also get feedback from people, tiktok, instagram.
Re: Rich Sutton on AI creativity and discovery
#75One has to be very specific when throwing around words like "creative" when talking about A.I Can A.I create art. Well it can create something that's pleasing to our senses but art is ultimately about conveying human feelings and emotions. Even as humans, understanding art is not universal. "feelings and emotions" and therefore art, can be deeply tied to a particular groups shared beliefs and experiences. Can it be c…
Current models are trained on image pastiche and style remixing. But there's no reason you couldn't add an Artistic Director layer which has been trained on emotional and cultural signifiers and to direct the pastiche and remixing. The practical problem is that models have very limited prompt adherence. The level of detail you can specify in scene design is very crude. So you can get the slop effect where there's a l…
>But there's no reason you couldn't add an Artistic Director layer which has been trained on emotional and cultural signifiers and to direct the pastiche and remixing.
I gave this approach a shot over the first few months of this year[1] (although my director didn't have any custom training). The results were interesting, but I'd not call them "art", since they're low-quality derivative pieces. With reasoning traces enabled, you can see that there's not much intent going on. Though they do attempt to include "incidental objects" to reinforce meaning, like in this jungle scene[2].
[1] https://news.ycombinator.com/item?id=48105385
[2]https://www.liamlaverty.com/paint-by-language-model/inspect/...
Re: Rich Sutton on AI creativity and discovery
#76One has to be very specific when throwing around words like "creative" when talking about A.I Can A.I create art. Well it can create something that's pleasing to our senses but art is ultimately about conveying human feelings and emotions. Even as humans, understanding art is not universal. "feelings and emotions" and therefore art, can be deeply tied to a particular groups shared beliefs and experiences. Can it be c…
> art is ultimately about conveying human feelings and emotions you made a small error, art is mostly about generating an emotion in the viewer/listener/.... not about transmitting an emotion of the creator the Wikipedia page on art starts with: > Art is a diverse range of cultural activity centered around works utilizing creative or imaginative talents, which are expected to evoke a worthwhile experience https://en.…
No art is a means of expression - the creator uses an object as a vehicle to capture and distribute an expression.
Also lol @ using Wikipedia as some source of absolute truth.
Re: Rich Sutton on AI creativity and discovery
#77One has to be very specific when throwing around words like "creative" when talking about A.I Can A.I create art. Well it can create something that's pleasing to our senses but art is ultimately about conveying human feelings and emotions. Even as humans, understanding art is not universal. "feelings and emotions" and therefore art, can be deeply tied to a particular groups shared beliefs and experiences. Can it be c…
Current models are trained on image pastiche and style remixing. But there's no reason you couldn't add an Artistic Director layer which has been trained on emotional and cultural signifiers and to direct the pastiche and remixing. The practical problem is that models have very limited prompt adherence. The level of detail you can specify in scene design is very crude. So you can get the slop effect where there's a l…
Sounds like a skill issue?
Recent image models are advancing rapidly at prompt adherence specifically, and being able to iterate on the same image is propelling them even further. Images 2.0 being the poster child of this "agentic iterative image composition" approach.
Re: Rich Sutton on AI creativity and discovery
#78The world will not be satisfied until we have read, and discussed every half-famous person’s opinion on AI. Still about ten million discussions to go.
He is a Turing award winner
Re: Rich Sutton on AI creativity and discovery
#79The most successful applications like coding are not the result of pure LLM/generative modeling. They come from closing the loop with an agentic harness. The generate-test-selectively refine loop is the core modality of scientific work. An LLM + RL with Verifiable Rewards + feedback from compiler/terminal runs mimics this process to a great extend. This is Fisher/Box feedback loop ( https://www-sop.inria.fr/members/I…
Completely agree on the importance of the harness. The problem I see is the same problem Evolutionary Algorithms had: you can generate potential solutions until you run out of cash, but you still need to evalulate those solutions. You need a fitness function, and that means you need to at least know the general shape of the solution. If anyone knows of any work towards more open-ended fitness functions, I'd love to r…
The TRM architecture models both the problem and the solution at the same time. You might find it an interesting read.
Re: Rich Sutton on AI creativity and discovery
#80Earlier quoted context omitted.
Current models are trained on image pastiche and style remixing. But there's no reason you couldn't add an Artistic Director layer which has been trained on emotional and cultural signifiers and to direct the pastiche and remixing. The practical problem is that models have very limited prompt adherence. The level of detail you can specify in scene design is very crude. So you can get the slop effect where there's a l…
Sounds like a skill issue? Recent image models are advancing rapidly at prompt adherence specifically, and being able to iterate on the same image is propelling them even further. Images 2.0 being the poster child of this "agentic iterative image composition" approach.
Images 2.0 isn't anywhere close to the kind of detail control I'm talking about.
It's the opposite of a skill issue. No image generator is anywhere near the ballpark of pro-level manual Photoshop or Illustrator editing for individual elements in an image.
If you don't understand this, try precisely kerning the text in a generated book cover to handle letter combinations like A and V.
This is one of the big problems with GenAI. You can do new things with it, but it's crude Dunning Kruger good-enough-if-you-don't-ask-for-more creativity.
The pros can see what most people can't, and the flaws and missing features are frustrating and obvious creatively, not just in terms of production values.