Rich Sutton on AI creativity and discovery

Yes, the guy with a PhD in Machine Intelligence, co-author of Reinforcement Learning: An Introduction, which is universally considered the bible of the field, recipient of the AAAI fellowship award and the Turing Award, and the inventor of Temporal Difference Learning doesn't know what he's talking about.

(For others like me who need to look it up.)

Stochastic: describes a process, model, or system that involves inherent randomness, meaning its future states or outcomes cannot be predicted with absolute certainty but can be described using probability distributions.

I don't completely disagree but its worth noting how new a lot of the empirical evidence in favour of LLMs are, so its not impossible to be a tad ignorant of the present

“When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.”

https://en.wikipedia.org/wiki/Clarke%27s_three_laws

There seems to be a problem with how he poses the problems alphaGo and these GAI models face.

AlphaGO is given a hard evaluation externally. It did not itself come up with it.

When GAI models are given an external hard evaluation, they can also succeed in many different domains (that is one of the remarkable features, succeeding in many domains) ranging from simple programming tasks to frontier mathematics (disproving conjectures recently) to writing more optimized kernel code than before.

And there is plenty of RL especially in these fields where the solution may be extremely complex but eval is rather less complex. And even the discovery and the "evolution-like" trace-selection is also happening.

For this reason it seems strange to compare it to AlphaGO as alphago is given a hard eval independent of itself, from an external source (humans) in a narrow domain. If GAI is given such, it can also show some remarkable results.

But what I find more strange is that innovation and moving forward in many many many cases does not require truly novel ideas but instead a high-quality execution of layering different methods, tactics, ideas on top of each other. Because in many domains our collective knowledge is incredibly sparse and complex, something being able to recombine tools, models, ideas in a high quality way (as he mentions being selective) I think is extraordinarily powerful. And in such cases, with a finite exploration horizon (time, resource available) with 1% "good choices" vs 3% "good choices" are worlds apart, incomparable.

Most importantly: none of the above is about intelligence, it's barren solution-farming to important, valuable problems we have. Most of the AGI and intelligence-related debate seems to miss out on this simple fact. (Insert the usual stuff like a plane being unable to fly like a bird or a submarine not swimming is totally irrelevant to it being useful).

And then a final point: do we really think this thing is incapable of doing better on average on problems we average people face in our lifetime? What should we think, how should we define human intelligence when we give out degrees in science or medicine for 60-70% exam results on problems considered to be generic in the field?