Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rich Sutton on AI creativity and discovery

twitter.com

Post reply on HN
Print view View original

101–110 of 141 posts

Re: Rich Sutton on AI creativity and discovery

#101

Post by lioeters »

oulipo2 wrote: post #60

I think that creativity / intelligence / humour is indissociable from "making errors". A joke is just an "error" - your brain predicted something, and the butt of the joke goes in another direction, and it's the mismatch that makes it funny. The same goes with creativity, and intelligence. The problem is that, by design, while trying to make machines "reliable", we make it impossible for them to be intelligent and cr…

In addition to the importance of making "mistakes", I would say "surprise" is a big element in creativity and humor. Perhaps these are related concepts, because a mistake is surprising. It's an unexpected departure from the normal routine and habit of behavior or thinking.

The term "surprisal" is used in information theory:

> For a given probability space, the measurement of rarer events are intuitively more "surprising", and yield more information content than more "common" events.

Can a machine surprise us? Given enough complexity, I think so. They can produce unpredictable results, even novelty, something we've never seen before. But does that mean a machine can be creative? Or funny? Maybe there's a threshold of acceptance, where eventually its output will become surprising enough that we might as well call it creative.

Re: Rich Sutton on AI creativity and discovery

#102

Post by lioeters »

dang wrote: post #27
habitue wrote: post #3

[flagged]

" Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something. " https://news.ycombinator.com/newsguidelines.html

That reminds me of a quote by Freeman Dyson on Stephen Wolfram's work.

> "There's a tradition of scientists approaching senility to come up with grand, improbable theories. Wolfram is unusual in that he's doing this in his forties."

I always felt it was an unfair dismissal of someone's life's work. Maybe it was true but it didn't enrich the discussion or our understanding. I suppose it means even a respected thinker can be guilty of shallow dismissals and saying hurtful things in public about others.

It's similar to a "thought-terminating cliché", in that it just reinforces an existing opinion without adding anything, making us think deeper, or furthering the conversation.

Re: Rich Sutton on AI creativity and discovery

#104

Post by visarga »

musebox35 wrote: post #52

The most successful applications like coding are not the result of pure LLM/generative modeling. They come from closing the loop with an agentic harness. The generate-test-selectively refine loop is the core modality of scientific work. An LLM + RL with Verifiable Rewards + feedback from compiler/terminal runs mimics this process to a great extend. This is Fisher/Box feedback loop ( https://www-sop.inria.fr/members/I…

> There are more elements to discovery though. It is still not clear where the initial working model/hypothesis comes from or how the updates are selected

That is a problem in RL, so we usually do supervised training first, teach it to imitate some trajectories, then do RL to refine the model. RL alone has a huge problem because it might be hard to reach a reward, hence hard to learn the task by pure reinforcement. Humans also combine supervision (learn from books) with search (solving problems) to break the discovery problem. For example, a human with no initial instruction in math would not produce great results no matter how smart they are. The bootstrap was exploration paid for in the past.

Re: Rich Sutton on AI creativity and discovery

#105

Post by YeGoblynQueenne »

balazstorok wrote: post #19

There seems to be a problem with how he poses the problems alphaGo and these GAI models face. AlphaGO is given a hard evaluation externally. It did not itself come up with it. When GAI models are given an external hard evaluation, they can also succeed in many different domains (that is one of the remarkable features, succeeding in many domains) ranging from simple programming tasks to frontier mathematics (disprovin…

>> (Insert the usual stuff like a plane being unable to fly like a bird or a submarine not swimming is totally irrelevant to it being useful).

Just a brief reminder that planes have wings with airfoils just like birds and submarines have air tanks just like fish have swimming bladders.

Some birds fly without flapping their wings much, too, e.g. albatrosses.

Re: Rich Sutton on AI creativity and discovery

#106

Post by visarga »

flir wrote: post #59
musebox35 wrote: post #52

The most successful applications like coding are not the result of pure LLM/generative modeling. They come from closing the loop with an agentic harness. The generate-test-selectively refine loop is the core modality of scientific work. An LLM + RL with Verifiable Rewards + feedback from compiler/terminal runs mimics this process to a great extend. This is Fisher/Box feedback loop ( https://www-sop.inria.fr/members/I…

Completely agree on the importance of the harness. The problem I see is the same problem Evolutionary Algorithms had: you can generate potential solutions until you run out of cash, but you still need to evalulate those solutions. You need a fitness function, and that means you need to at least know the general shape of the solution. If anyone knows of any work towards more open-ended fitness functions, I'd love to r…

> If anyone knows of any work towards more open-ended fitness functions, I'd love to read it.

There is research in open-ended learning, see "Why Greatness Cannot Be Planned" by Kenneth O. Stanley. The core idea is that in open-ended scenarios you don't know what action was good except in hindsight because your path is deceptive. So the idea is to replace fitness with novelty search which provides more stepping stones towards the goal.

Re: Rich Sutton on AI creativity and discovery

#107

Post by pixl97 »

chadgpt3 wrote: post #93
redblacktree wrote: post #90

Earlier quoted context omitted.

We do have a LOT of water though. And surplus helium would be nice.

We also have competitive energy wasting (bitcoin), that will translate into competitive water deleting which will greatly accelerate how quickly we burn through all of it.

Eventually we'll burn up the visible universe, but I think we have some time before then.

Re: Rich Sutton on AI creativity and discovery

#108

Post by visarga »

premieroncall wrote: post #91

Hmm, so the guy who said more data and compute will outperform any adhoc heuristics has shared a three step adhoc heuristic?

The ad hoc heuristics are the domain knowledge baked into the model by human experts, like features, architecture and loss function.

"Evaluation" means environments or datasets, the model is supposed to discover its representations from scaled up experience. That was the bitter lesson - more data and compute beat heuristics.

Re: Rich Sutton on AI creativity and discovery

#109

Post by pixl97 »

rembicilious wrote: post #14

"So that is my call to arms. If we want the full power of AI scientists, then we should share the goals with them so they can create, evaluate, discover, and in these ways fully participate in achieving the goals. Let’s be bold! Let’s fully automate Creativity and Discovery!" Should we automate exercise and play as well? How about learning? The machine didn't have a soul, so we donated ours. Eureka! My AI found it!

Most discovery takes a ton of iteration and repetition. That's a lot less fun than the hedonism of sitting on a beach drinking a mahi thai.

Same with learning, humans historically where generalists without that deep of knowledge when compared to one another. Now we study a quarter of our short lives just to get to the point where we can specialize for the rest of our lives. This situation doesn't seem exactly tenable as complexity increases in the future.

I don't know what the future looks like, but I can tell you that entropy and complexity will only increase.

Re: Rich Sutton on AI creativity and discovery

#110

Post by anthonypasq »

musebox35 wrote: post #52

The most successful applications like coding are not the result of pure LLM/generative modeling. They come from closing the loop with an agentic harness. The generate-test-selectively refine loop is the core modality of scientific work. An LLM + RL with Verifiable Rewards + feedback from compiler/terminal runs mimics this process to a great extend. This is Fisher/Box feedback loop ( https://www-sop.inria.fr/members/I…

Yes it seems most anti-LLM researchers take issue with LLMs on fundamental math/architecture based properties, but seem to miss all the engineering going on around the model to make it useful.

Those mathematical shortcomings very well might mean they arent a path to true AGI, but that honestly seems fairly irrelevant at this point tbh.

Post reply on HN