A Reply to François Chollet on Intelligence Explosion

In my opinion, the historical record from, say, 1517 (or even 1717) is so full of gaps and inconsistencies that it is impossible to make even an order of magnitude level estimate regarding something like rate of technological innovation (or even world GDP, arguably).

Economists and social scientists are often guilty of this as well - for instance, I really liked the parts of Pinker's Better Angels of Our Nature that dealt with relatively reliable quantitative data from the 20th and 19th centuries. But once he starts talking about things like the An Lushan rebellion or the fall of the Roman Empire, it's a complete mess from an empirical point of view, doing things like taking the numbers of fatalities cited by contemporary participants in an historical conflict at face value. No self-respecting historian would make such big assumptions from such faulty data.

Anyway, I realize this is incidental to the larger argument here but insofar as questions about exponential growth of technology draw on historical arguments, I wanted to throw it out there.

The google search result page's source code is, for many individual humans, already so complex as to be beyond understanding. And that's a computational artifact largely produced by other humans directly!

Say you build an AI, and ask it how to win a political election - and it outputs a simple list of reasonable-sounding suggestions of where to campaign, promises to make, people to meet, slogans to use, and criticisms of your opponent to focus on.

Before actually implementing those suggestions, do you think you could be _very_ certain that following those suggestions would result in you winning the election? Or, would it be possible that the AI understood social dynamics so much better, that it gave you a list of instructions that seemed mostly reasonable, but actually result in your opponent winning in a landslide? Or the country undergoing revolution? Or, you winning, along with a surprising social trend of support for funding AI research?

Look at the time frame from from the invention of controlled heavier-than-air flight to landing a man on the moon; within a human lifetime and before you were born.

Look at the time frame from the point when the vast majority of the human population lived their entire life within a 50 mile radius of where they were born and when fast transportation and global travel exponentially increased the genetic mixing of humanity; within a human lifetime and before our grandparents were born.

Look at the time frame from the point when information could travel no faster than the speed of a good horse to the time when information could travel across the ocean in the time it took you to saddle a horse; within a human lifetime and almost two centuries ago.

Our progress in the last century is significant, but you over-estimate its importance because you are surrounded by it and have little understanding of the history of technology. Things that may seem trivial or even primitive to you were far more important and world-changing inventions, while a lot of what we currently consider significant advances are only important because, for example, we lived in a time when people played finite games better than machines and were around to see that era end.

Someone born in 1900, looking back at their life in 1975, would be like "When I was born, heavier-than-air flight was impossible. Now people routinely fly across oceans at 600 mph, you can travel faster than the speed of sound for admittedly a lot of money, and we've gone to the motherfuckin' moon. Vast swathes of work has been automated, to the point where we essentially ended an entire industry (personal servants). Automobiles went from being curiosities to something that even poor people have and use every day. We split the atom, we brought women into the workforce, we invented electronic computers, we invented radar, we turned radio from a science project to TVs that every family have. We invented antibiotics and childhood mortality fell by some enormous percentage."

"You're very impressed that computers went from 'pretty good at playing chess' to 'extremely good at playing chess' in just 20 years. Maybe you're the one who's missing the forest for the trees."