Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rich Sutton on AI creativity and discovery

twitter.com

Post reply on HN
Print view View original

111–120 of 141 posts

Re: Rich Sutton on AI creativity and discovery

#111

Post by anthonypasq »

visarga wrote: post #104
musebox35 wrote: post #52

The most successful applications like coding are not the result of pure LLM/generative modeling. They come from closing the loop with an agentic harness. The generate-test-selectively refine loop is the core modality of scientific work. An LLM + RL with Verifiable Rewards + feedback from compiler/terminal runs mimics this process to a great extend. This is Fisher/Box feedback loop ( https://www-sop.inria.fr/members/I…

> There are more elements to discovery though. It is still not clear where the initial working model/hypothesis comes from or how the updates are selected That is a problem in RL, so we usually do supervised training first, teach it to imitate some trajectories, then do RL to refine the model. RL alone has a huge problem because it might be hard to reach a reward, hence hard to learn the task by pure reinforcement. H…

the role of evolution is always a confounding factor as well and all the various analogies to how it maps onto AI research are always not quite satisfactory.

Re: Rich Sutton on AI creativity and discovery

#112

Post by anthonypasq »

nateroling wrote: post #9
edot wrote: post #4

I don't quite follow his point. Is it: a) that we need a new foundational algorithm that integrates a goal (one with "taste") directly into the training step, or b) that we need to point trained models towards goals as they iterate? If it's a), he doesn't propose such an algorithm, and I don't know how you'd do it at such a low level because how do you quantify abstract goals? Did he suggest such an algorithm and I m…

No, I think I he’s saying that we have that, and we should use it more. AlphaGo uses discovery when it evaluates potential moves and iterates. Claude Code uses discovery when it generates a script and the evaluates whether it works or not. He’s saying we need to allow ai systems to do the evaluation and iteration themselves for science and engineering the same way we do for code. Basically, harness engineering for en…

but this has to happen during training no?

Re: Rich Sutton on AI creativity and discovery

#113

Post by dang »

premieroncall wrote: post #91

Hmm, so the guy who said more data and compute will outperform any adhoc heuristics has shared a three step adhoc heuristic?

Can you please make your substantive points thoughtfully and without snark? I'm sure there is one here, but it's hard to make out what it is, and in any case the poison does more harm than the information does good.

I'd link to the HN guidelines here but I'm on my phone!

Re: Rich Sutton on AI creativity and discovery

#114

Post by antonvs »

musebox35 wrote: post #56
rembicilious wrote: post #14

"So that is my call to arms. If we want the full power of AI scientists, then we should share the goals with them so they can create, evaluate, discover, and in these ways fully participate in achieving the goals. Let’s be bold! Let’s fully automate Creativity and Discovery!" Should we automate exercise and play as well? How about learning? The machine didn't have a soul, so we donated ours. Eureka! My AI found it!

I understand the skepticism. I am worried about the implications of AI as well. The deeper issue at stake is that the depth of scientific knowledge has been increasing for a very long time. Now you get to have a PhD in esoteric subproblems and that slows down research especially if the discoveries require depth in multiple subdomains. Socially and economically training people in every combinatorial combination of sub…

Commercial fusion energy is almost certainly a pipe dream. We already have good alternatives. Short of some unpredictable major breakthrough in physics on the level of something like cold fusion, the current activity in the fusion space will basically just serve to demonstrate definitively that it's not viable.

Re: Rich Sutton on AI creativity and discovery

#115

Post by amarant »

tines wrote: post #87
rembicilious wrote: post #14

"So that is my call to arms. If we want the full power of AI scientists, then we should share the goals with them so they can create, evaluate, discover, and in these ways fully participate in achieving the goals. Let’s be bold! Let’s fully automate Creativity and Discovery!" Should we automate exercise and play as well? How about learning? The machine didn't have a soul, so we donated ours. Eureka! My AI found it!

It seems human beings spend a significant portion of their lives trying to figure out how to live as little as possible.

They do say "live a little", and it strikes me now that perhaps I've completely misinterpreted that phrase. For some reason I've always thought it to be a call to live a lot, or at least more.

But now I see that my interpretation is almost opposite of what the words mean... Huh...

Re: Rich Sutton on AI creativity and discovery

#116

Post by musebox35 »

visarga wrote: post #104
musebox35 wrote: post #52

The most successful applications like coding are not the result of pure LLM/generative modeling. They come from closing the loop with an agentic harness. The generate-test-selectively refine loop is the core modality of scientific work. An LLM + RL with Verifiable Rewards + feedback from compiler/terminal runs mimics this process to a great extend. This is Fisher/Box feedback loop ( https://www-sop.inria.fr/members/I…

> There are more elements to discovery though. It is still not clear where the initial working model/hypothesis comes from or how the updates are selected That is a problem in RL, so we usually do supervised training first, teach it to imitate some trajectories, then do RL to refine the model. RL alone has a huge problem because it might be hard to reach a reward, hence hard to learn the task by pure reinforcement. H…

SFT + RL connection to model/hypothesis search is insightful. Brute force / scalable search is where Sutton's Bitter Lesson also points to. Once your search domain is small compared to your search budget, that makes a lot of sense.

If I get your meaning right, SFT creates the right inductive bias so that the RL search + reward guidance does the trick.

For novel discovery, the question might then be whether the inductive bias builds a strong enough prison so no new discovery is possible by RL or if the search can escape the boundaries set by SFT given enough randomization and the right reward function.

I know that RL is usually not performed at inference time, but in-context learning mechanisms might be developed by RL to discover at test time. Edit: I would love to hear if that actually happens or not, like new induction heads (https://transformer-circuits.pub/2022/in-context-learning-an...) forming during RL. I really have no idea.

Re: Rich Sutton on AI creativity and discovery

#117

Post by volkercraig »

Shitty-kitty wrote: post #30

One has to be very specific when throwing around words like "creative" when talking about A.I Can A.I create art. Well it can create something that's pleasing to our senses but art is ultimately about conveying human feelings and emotions. Even as humans, understanding art is not universal. "feelings and emotions" and therefore art, can be deeply tied to a particular groups shared beliefs and experiences. Can it be c…

You can create art with everything from sticks and mud to glass and air. Of course you can make art with AI.

Now if the question is, can a machine make art, well ultimately someone needed to turn the machine on and design the machine to make art, so arguably that person/people are the ones making the art.

Historically, every question of "is x art" ends up having the answer "yes". I don't know why people fall for the same thing over and over.

Re: Rich Sutton on AI creativity and discovery

#119

Post by simianwords »

arowthway wrote: post #65
simianwords wrote: post #36

Earlier quoted context omitted.

That’s my point, he says ai does well where evaluation is neurosymbolically closed. But so do humans? How do humans make discoveries without having formal ways to evaluate? In my pharma drug example, humans could evaluate only because they had access to the physical realm. I can’t think of an example of humans evaluating a discovery in a way that LLMs can’t. can you?

I don't think there is any "humans are metaphysically superior to LLMs" subtext to this talk, it's just a technical/educational observation. Access to some forms of evaluation and selective retention is inherent to humans and it's not inherent to LLMS. But it can be somehow bolted on and that's when they work best. It makes sense that more focus on those principles can yield better AI. I think the retention part is t…

> Access to some forms of evaluation and selective retention is inherent to humans and it's not inherent to LLMS

I'm not sure I understood - what forms of evaluation is inherent to humans? If you don't give humans tools or access to the physical world, how can they evaluate?

Re: Rich Sutton on AI creativity and discovery

#120

Post by AndrewKemendo »

dwd wrote: post #10

"We have many AI systems which can give us more. ... and Claude-Code, which have brought true advances in science, mathematics, and programming." That contradiction kind of says he doesn't know what he's talking about.

You know it’s occurred to me recently that there really is no value in this age to any kind of professionalism or seniority

If you don’t agree with somebody, nothing else matters

It’s like people (you as an example) have taken the concept of experts and fucked it up so bad that simultaneously everybody thinks they’re an expert while also dismissing everybody else who claims they are an expert

It’s like the whole concept is entirely poisoned. Worse everyone is smugly pointing at the Wikipedia for “appeal to authority.”

Nothing new I suppose, Socrates after all was driven to suicide by the madness of his society accusing him of impiety.

Post reply on HN