Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rich Sutton on AI creativity and discovery

twitter.com

Post reply on HN
Print view View original

21–30 of 141 posts

Re: Rich Sutton on AI creativity and discovery

#21

Post by kibibu »

highfrequency wrote: post #18

Unless I'm missing something, this argument seems to apply only to the original pretraining era (eg GPT 1-4). The post-training and reinforcement learning paradigms are clearly doing variation, evaluation and selective retention no?

The transcript does seem to overlook post-training steps like Reinforcement Learning with Verifiable Rewards (RLVR) (but I'll certainly won't claim that Rich Sutton is unaware of such things; RLVR has a very narrow set of evaluation approaches).

I wonder if this is a precursor to Keen Tech leaning into David Silver's Ineffable Intelligence approach.

Re: Rich Sutton on AI creativity and discovery

#22

Post by thedreammachine »

Humm maybe. But a plain model sampling outputs obviously isn't doing discovery in the AlphaGo sense. But once you put the model in a loop with tests, feedback, tools or even a human picking the good result, it starts to get much closer to the process he's describing.

Re: Rich Sutton on AI creativity and discovery

#23

Post by dwd »

phyzix5761 wrote: post #11
dwd wrote: post #10

"We have many AI systems which can give us more. ... and Claude-Code, which have brought true advances in science, mathematics, and programming." That contradiction kind of says he doesn't know what he's talking about.

Yes, the guy with a PhD in Machine Intelligence, co-author of Reinforcement Learning: An Introduction, which is universally considered the bible of the field, recipient of the AAAI fellowship award and the Turing Award, and the inventor of Temporal Difference Learning doesn't know what he's talking about.

Sure, but does that mean he's right all the time about all things, including everything in his own field?

He is saying no generative AI is going to produce output that is both good and novel because it is always derivative. And then adds a generative AI (Claude Code) into his list of AI that have produced output that he feels is good and novel, invalidating what he is arguing.

"...no matter how many instances of white swans we may have observed, this does not justify the conclusion that all swans are white."

Re: Rich Sutton on AI creativity and discovery

#24

Post by armchairhacker »

I don’t think ML can’t be creative or make discoveries. I think creativity and discovery are, ultimately, simultaneously thinking about the right seemingly-disparate concepts (whereas algorithmic thinking is more obviously related concepts). If not an LLM, some other model can generate random ideas, rank them, then output the best.

But I think humans are better at it, while ML is better at algorithmic thinking. “Better” being more efficient and something we more enjoy doing; we can also more accurately rank what subjectively appeals to humans (i.e. taste), especially ourselves.

I think ML should be optimized for tasks that require more generalization than programming, but are still mostly logic. Like software development, translation, and tools for art and discovery.

Re: Rich Sutton on AI creativity and discovery

#25

Post by simianwords »

I'm trying to keep an open mind and understand what the author is trying to say because he is credentialed.

His main point is that discoveries involve

1. Variation,

2. Evaluation, and

3. Selective retention.

He makes a jump saying AI is only capable of 1) and humans are capable of 1) 2) and 3). I don't know what makes humans special enough that they can do 2) and 3)?

In fact, the more you think of this it is kind of strange - in science humans can only do "evaluation" because they have access to the real world. They can evaluate a new drug because they can do it on people so it is not some inherent limitation of AI but rather access to physical realm.

Finally I want to ask a specific thing: how do you mathematically falsify what this person is saying? How can you formally prove that - no AI can not "evaluate"? I ask because I make AI evaluate a lot of people's claims and it works for me.

Re: Rich Sutton on AI creativity and discovery

#28

Post by cortesoft »

> When we ask for a fiction or novelty, the AI can give it to us because its processing is in part stochastic. Every decision can go multiple ways and will go different ways and produce a different trajectory every time. The trajectory can be random—and thus novel—or it can be based on the training data—and thus “good” because the training data is good, sourced from people or reality. Thus, the trajectory is either novel or good—based on randomness or based on data—but never both at the same time.

This doesn't seem true? You can be both random and based on training data.

Re: Rich Sutton on AI creativity and discovery

#29

Post by LarsDu88 »

kibibu wrote: post #21
highfrequency wrote: post #18

Unless I'm missing something, this argument seems to apply only to the original pretraining era (eg GPT 1-4). The post-training and reinforcement learning paradigms are clearly doing variation, evaluation and selective retention no?

The transcript does seem to overlook post-training steps like Reinforcement Learning with Verifiable Rewards (RLVR) (but I'll certainly won't claim that Rich Sutton is unaware of such things; RLVR has a very narrow set of evaluation approaches). I wonder if this is a precursor to Keen Tech leaning into David Silver's Ineffable Intelligence approach.

This was exactly what I was thinking of. RLVR is the secret sauce behind o3 and its many successors.

Its the secret sauce behind why the current models are so great at coding and soon to be unbeatable at math.

LLMs can pose many questions and if they are easily verifiable, fine tune very heavily. A lot of the world models discussion will inevitable lean into simulations as verification.

Re: Rich Sutton on AI creativity and discovery

#30

Post by Shitty-kitty »

One has to be very specific when throwing around words like "creative" when talking about A.I

Can A.I create art. Well it can create something that's pleasing to our senses but art is ultimately about conveying human feelings and emotions. Even as humans, understanding art is not universal. "feelings and emotions" and therefore art, can be deeply tied to a particular groups shared beliefs and experiences.

Can it be creative in non-subjective fields such as math or sciences. Einstein derived GR from his creative thought experiments. If an A.I poped out GR's field equations simply by testing different mathematical frameworks that resolve the issues discovered by experiments, is that creative? Perhaps but certainly not in the same way.

Post reply on HN