DeepMind: A Generalist Agent

> concerns about AGI make about as much sense as an overpopulation on Mars

I disagree strongly that this is an apt analogy. Planning strategies for dealing with overpopulation on Mars is contrived and unnecessary, whereas planning for AGI is more reasonable.

The creation of AGI is a more important event than overpopulation of any given planet. There is good reason to believe that mishandling the creation of AGI would pose a permanent existential threat to humans. Overpopulation on Mars would only be an existential threat if we believed it to be followed by an exhausting of resources leading to extinction of all humans in our solar system. It is contrived to worry about that now.

There is no good way to know just how close or far we are from AGI like there would be to predict overpopulation on Mars. In general, we have a strong grasp on the fundamental dynamics of overpopulation, whereas we don't yet have a strong grasp on how intelligence works.

People have been very bad at predicting when AI would be capable of accomplishing tasks. There have been many under- and over- estimates by prominent researchers. If progress is unpredictable, there is some significant chance we are closer to AGI than most people think.

AGI is both far more important and more probable than overpopulation of Mars in the next 20 years.

> But I don't think there is any indication that they are moving towards human level intellect at some exponential rate.

Is there any very strong indication that progress is plateauing, or that the current approach of deep learning is definitely not going to work? If your benchmark is simply "can it do X, or not?", it's not a very good benchmark for determining progress. That's why benchmarks usually have scores associated with them.

> If DALL-E suddenly started to discuss philosophy with me I'd be concerned

If DALL-E suddenly started discussing philosophy with you in a way that would concern you in that moment, you should have been concerned for years.

Playing Atari is cool, but it's just another "trick". Training a computer to do progressively more difficult tasks doesn't seem much more impressive than training an animal to do so.

I see no evidence in the paper that it can learn arbitrary tasks on the fly. It's very impressive, though.

I first got involved in text classification in the early 00's and then the best you could do was "bag of word" models that counted the words in a document but didn't take the order of words into account.

This works great if you asking a question "Is this paper about astrophysics?" because the vocabulary used in a document is closely linked to the topic.

Pretty obviously though if you scramble the words in the document you can't reconstruct the original document, some information is lost, and there are some classification tasks that will reach an upper limit (asymptote) in accuracy because in taking the feature set you lost something. (If the task is "did the defendant commit the crime" the heuristic "Tyrone is a thug" works over bag-of-words, but there is no justice in that.) If that system is able to get the right answer for a case where the word order matters, it just got lucky.

You might think "wouldn't it be better to use pairs of words?" but then you run into another problem. You might have a vocabulary of 2,000-20,000 words and get a somewhat useful sample of all of those in a few thousand documents. The number of word pairs is the square of the number of words and you just can't get enough training samples to sample all the possible word pairs.

Sentiment analysis was an early area where bag-of-words broke down because

   I am happy

and

   I am not happy

mean very different things. You'd think now that adjectives like "happy" really are special and so is the word "not" and we could make the system somehow realize that "not X" means the opposite of X. You run into an asymptote situation there because there are a huge number of possible negation patterns, for instance you can say

   I can't say that I am happy

and you can't even say "the negation structure has to be within ten words of the adjective" because there is no limit for how complex nested structures can get in language. The first few patterns you add "not X" raise the performance potential of the system a lot but patterns you add after that each make a smaller and smaller contribution to the performance and you again reach an asymptote.

Today we have all kinds of embeddings and they are a step forward but they also run into the risk of throwing critical information away, and in a multi-step system you are doomed if an early step does that. I've walked away from some projects where people required high accuracy and they were stuck on using word embeddings that would never attain it. You can think about information loss in embeddings the same way as you do with simpler features except it is a lot more complicated and a lot of people look away instead of confronting the problem.

It might be basically the same process as today but with several big new ideas (some of which might seem simple in retrospect...)

The quality of the training set is also critical, more so than the quantity. Some of these clever ideas for creating a lot of training data without any work, such as "guess the next word" can't really capture semantics.

I think it really takes multi-task training, like what the article we are talking about is advocating. That forces the upstream part of the network to learn features that capture important semantics.

I think the best-case scenario is that 'we' become something different than we are right now. The natural tendency of life(on the local scale) is toward greater information density. Chemical reactions beget self-replicating molecules beget simple organisms beget complex organisims beget social groups beget tribes beget city states beget nations beget world communities. Each once of these transitions looks like the death of the previous thing and in actuality the previous thing is still there, just as part of a new whole. I suspect we will start with natural people and transition to some combination of people whose consciousness exists, at least partially, outside of the boundaries of their skulls, people who are mostly information on computing substrate outside of a human body, and 'people' who no longer have much connection with the original term.

And that's OK. We are one step toward the universe understanding itself, but we certainly aren't the final step.