[dead]
Rich Sutton on AI creativity and discovery
41–50 of 141 posts
Re: Rich Sutton on AI creativity and discovery
#42I'm trying to keep an open mind and understand what the author is trying to say because he is credentialed. His main point is that discoveries involve 1. Variation, 2. Evaluation, and 3. Selective retention. He makes a jump saying AI is only capable of 1) and humans are capable of 1) 2) and 3). I don't know what makes humans special enough that they can do 2) and 3)? In fact, the more you think of this it is kind of…
He's saying that pre-training an LLM alone can't do it, but if you run an LLM in a loop with tools (like any coding agent) then it can. Also, the technique his group came up with should be used more: > This is the weakness of deep learning that is alleviated with a new algorithm that my group presented in Nature a couple of years ago. Our “continual backpropagation” made one small change: every so often a less-used n…
Sorry this makes no sense — humans also use tools to evaluate their discoveries.
Kant said something like this: knowledge can’t be obtained by pure thinking, it needs interaction with the world.
This is obvious to me so why is the author making a claim that LLMs can make knowledge without access to environment but purely through thinking in aether
Re: Rich Sutton on AI creativity and discovery
#43> When we ask for a fiction or novelty, the AI can give it to us because its processing is in part stochastic. Every decision can go multiple ways and will go different ways and produce a different trajectory every time. The trajectory can be random—and thus novel—or it can be based on the training data—and thus “good” because the training data is good, sourced from people or reality. Thus, the trajectory is either n…
I think they meant more "it can be extrapolated or interpolated" or "it can be high variance and 'creative' or it can be low variance and 'reliable/correct/likely'". If you want to see something new, the model will need to step off the manifold. But the manifold is where you've learned the "correct" solutions live.
Re: Rich Sutton on AI creativity and discovery
#44It’s ok, LLMs are useful as they are today. Even if they can never can come up with the next generation of math, physics etc.
Even for humans the brains who managed a step change in thinking are so rare that we literally know them by name.
Re: Rich Sutton on AI creativity and discovery
#45It’s ok, LLMs are useful as they are today. Even if they can never can come up with the next generation of math, physics etc. Even for humans the brains who managed a step change in thinking are so rare that we literally know them by name.
You might be missing that those rare humans were sitting on tons of failed or somewhat useful discoveries made by more “mediocre” humans that history forgot.
Re: Rich Sutton on AI creativity and discovery
#46It's funny to me that one would rate their own takes as "new and possibly controversial". Whatever comes next is read under that light of an author that thinks this about their own thoughts.
And the core point is not even true. They can definitely output novel things that are good - less so but they can and they do. Plenty of examples.
> Thus, the trajectory is either novel or good—based on randomness or based on data—but never both at the same time.
This assumes no possible unexplored path yields good results, or said another way, that none of the random results can be good, which is not true. The whole text seems to try to prove a point decided a-priori rather than make a case based on reality.
Re: Rich Sutton on AI creativity and discovery
#47"We have many AI systems which can give us more. ... and Claude-Code, which have brought true advances in science, mathematics, and programming." That contradiction kind of says he doesn't know what he's talking about.
Surprisingly enough, Turing Award winner and father of reinforcement learning Richard Sutton knows perfectly well what he's talking about. The whole talk is about the need to have the ability to test novel outputs against reality and iterate to find ones that are good. This is exactly what Claude Code, the agent framework, adds to Claude, the LLM, to allow it to find novel coding solutions that actually work.
Re: Rich Sutton on AI creativity and discovery
#48[deleted]
Re: Rich Sutton on AI creativity and discovery
#49This reset my thoughts
Re: Rich Sutton on AI creativity and discovery
#50He seems to be saying that Claude Code can make discoveries. Does anyone think novel discoveries can be made from systems created by supervised learning only, and attempting to do so? > Claude-Code, which have brought true advances in ... programming. ... these systems have found things that are both novel and good.