AI reminds me a lot https://en.wikipedia.org/wiki/TRIZ , it is like a good machine implementation of it.
Rich Sutton on AI creativity and discovery
31–40 of 141 posts
Re: Rich Sutton on AI creativity and discovery
#32> When we ask for a fiction or novelty, the AI can give it to us because its processing is in part stochastic. Every decision can go multiple ways and will go different ways and produce a different trajectory every time. The trajectory can be random—and thus novel—or it can be based on the training data—and thus “good” because the training data is good, sourced from people or reality. Thus, the trajectory is either n…
Only to a limited extent.
Re: Rich Sutton on AI creativity and discovery
#33Earlier quoted context omitted.
Yes, the guy with a PhD in Machine Intelligence, co-author of Reinforcement Learning: An Introduction, which is universally considered the bible of the field, recipient of the AAAI fellowship award and the Turing Award, and the inventor of Temporal Difference Learning doesn't know what he's talking about.
Sure, but does that mean he's right all the time about all things, including everything in his own field? He is saying no generative AI is going to produce output that is both good and novel because it is always derivative. And then adds a generative AI (Claude Code) into his list of AI that have produced output that he feels is good and novel, invalidating what he is arguing. "...no matter how many instances of whit…
If you read it he says that CC has additional aspects beyond ordinary GAI, namely the ability to verify. That aspect is necessary for GAI to be good and novel.
Although personally I think code doesn’t actually need to be very novel so it’s actually the best example.
Re: Rich Sutton on AI creativity and discovery
#34I'm trying to keep an open mind and understand what the author is trying to say because he is credentialed. His main point is that discoveries involve 1. Variation, 2. Evaluation, and 3. Selective retention. He makes a jump saying AI is only capable of 1) and humans are capable of 1) 2) and 3). I don't know what makes humans special enough that they can do 2) and 3)? In fact, the more you think of this it is kind of…
He actually says the areas in which AI has had the novel successes are those which can be evaluated (like coding or Go). Not that it can’t happen at all.
Re: Rich Sutton on AI creativity and discovery
#35He seems to be saying that Claude Code can make discoveries. Does anyone think novel discoveries can be made from systems created by supervised learning only, and attempting to do so?
> Claude-Code, which have brought true advances in ... programming. ... these systems have found things that are both novel and good.
Re: Rich Sutton on AI creativity and discovery
#36I'm trying to keep an open mind and understand what the author is trying to say because he is credentialed. His main point is that discoveries involve 1. Variation, 2. Evaluation, and 3. Selective retention. He makes a jump saying AI is only capable of 1) and humans are capable of 1) 2) and 3). I don't know what makes humans special enough that they can do 2) and 3)? In fact, the more you think of this it is kind of…
He actually says the areas in which AI has had the novel successes are those which can be evaluated (like coding or Go). Not that it can’t happen at all.
That’s my point, he says ai does well where evaluation is neurosymbolically closed.
But so do humans? How do humans make discoveries without having formal ways to evaluate? In my pharma drug example, humans could evaluate only because they had access to the physical realm.
I can’t think of an example of humans evaluating a discovery in a way that LLMs can’t. can you?
Re: Rich Sutton on AI creativity and discovery
#37[deleted]
Re: Rich Sutton on AI creativity and discovery
#38I'm trying to keep an open mind and understand what the author is trying to say because he is credentialed. His main point is that discoveries involve 1. Variation, 2. Evaluation, and 3. Selective retention. He makes a jump saying AI is only capable of 1) and humans are capable of 1) 2) and 3). I don't know what makes humans special enough that they can do 2) and 3)? In fact, the more you think of this it is kind of…
He's saying that pre-training an LLM alone can't do it, but if you run an LLM in a loop with tools (like any coding agent) then it can. Also, the technique his group came up with should be used more:
> This is the weakness of deep learning that is alleviated with a new algorithm that my group presented in Nature a couple of years ago. Our “continual backpropagation” made one small change: every so often a less-used neuron would be re-initialized to small random weights. This allows the variation to continue and plasticity to be retained.
Here's the paper: https://www.nature.com/articles/s41586-024-07711-7
It has a fair number of citations, but I haven't looked into how much it's used.
Re: Rich Sutton on AI creativity and discovery
#39I think a lot of deep learning is compositional generalization. Models learn reusable pieces (abstractions, styles, procedures, constraints, etc) and recombine them in ways that may never have appeared as a whole in the training data. So even if the ingredients come from past data the final composition can still be novel in a meaningful sense
Re: Rich Sutton on AI creativity and discovery
#40> That is, I would say that creativity requires that the new things generated be Evaluated. Without evaluation, and retention of the best, there is nothing created. The novelty flickers into existence but, if its value is unrecognized, it flickers away and is lost.
I really like the way he frames this here. I think a lot of people in the twitter comments (and maybe a few here) aren't reading past the introduction. He isn't saying AI systems are incapable of creativity and discovery. He is claiming generative AI without a harness is not capable of creativity and discovery. There needs to be some other system that "recognizes the value" of the novel idea and remembers it. He gives examples of where this value recognition step is automated and thus by his definition achieve creativity and discovery in a fully automated system.