I gave this talk a while ago to a small group of attendees. It was not recorded (I saw some ask below). It's based on a document I wrote a while ago called "You suck at writing AI" (never published). The basic argument was that people are comically inadequate at writing complex code. You can't write the code to detect a cat in an image and the correct thing to do is to give up, write down an objective that measures t…
Plot twist: figuring out that objective function may prove as intractable as the original problem! We'll need an objective function writing objective function, and then it's turtles all the way down.
The main objective function in nature is very simple though: maximize the number of copies of your genes. Such an objective enforced in a resource-constrained multi-agent world (as suggested in the slides) could really lead to quite complex sub-objectives which may as well lead to general intelligence. For example, if each agent can process information and perform work,
it follows that individuals that have better cooperative abilities also have a larger reproductive success. Cooperation is, however, extremely complex: It requires communication, identification of other individuals, establishment of trust, early detection of betrayal etc. The necessity for modeling the actions of other agents alone provides plenty of correction signals toward general intelligence because modelling other agents is such a difficult task.