Why human intelligence and AI will co-evolve

He's forgetting two other scaling modes applicable to human intelligence: accumulating knowledge and collaborating. By accumulating knowledge we can build upon the work of the generations before us and take essentially unlimited time to crack a problem. Collaboration allows not only parallel processing of problems but also interacting and exchanging inspiration towards the solution of a single problem; both which scale quite well with a lot of intelligent people.

Note that there are almost trivial arguments showing some problems are beyond the means of our finite brain (viz unbounded algorithmic complexity or the halting problem).

Consider that the first artificial intelligence probably won't be (exactly) designed at all. It will be a neural network or some other construct of an advanced algorithm that's solving some machine learning problem, and once it emerges no single human being (or possibly even group of humans) will fully understand why it works. Lets say that this AI is, against all odds, smarter than any human being. Chances are that NO human being will understand how the AI works, but why would you think that the AI understands itself? Why would we be certain that an AI can understand it's own neural network better than humans can, and furthermore, be able to quickly iterate on it, especially if a chaotic and complex process produced it in the first place?

I suspect that machine intelligence will arise in a much more similar fashion to messy organic evolution than you think, and it will be subject to all the same disadvantages - lots of random chance, dead ends and very slow advancement by trial and error.

Imagine what you could do with a planet-wide human breeding and genetic modification program with millions of generations - that's actually feasible for an AI.

IBM had been building tabulators for decades. But they just added and subtracted. Mechanical desk calculators had been built that could multiply and divide. Those came together in the IBM 602A Calculating Punch of 1946. A multiply in only a few seconds! Division, too. You could even do Newton's method by wiring the plugboard appropriately.

The limits of gear-driven arithmetic having been reached, IBM tried using vacuum tubes, and produced the IBM 603 Multiplier. This was roughly equivalent to a 602A, but it used tubes. It was a trial to find out if tubes would work in a fielded product; only 100 were built. They did. So IBM went on to the 604, which was like a 603 with more registers and more program steps.

Meanwhile, crystal radios had been around for decades, and germanium diodes followed as a cleaned-up form of those diodes. Some experimenters had fooled around with 3-terminal solid state devices; Lilienfeld patented one in 1925. But until materials processing improved, nobody could make one consistently. Only when germanium diodes were badly needed for WWII radar was that materials problem solved. The transistor followed.

So IBM kept plugging along. Next was the IBM 608, which was sort of like a 604, but with transistors. Then came the 609, which was like a 608, but faster. There hadn't been any conceptual change from the gear and relay era, but the hardware was getting much better. All these machines used decimal arithmetic.

Meanwhile, magnetic recording was coming along. There were wire recorders in the 1930s, tape recorders in the 1940s, and by 1944, Ampex was making some good ones. The first digital tape drive was a project for Arlington Hall, a predecessor of NSA.

In the 1940s through the 1960s, there were many special purpose machines that were almost computers, but not quite. American Totalizator had machines for racetracks. (They later invested in UNIVAC). Teleregister had machines for stockbrokers, and later, the first airline reservation system, Reservisor. There were ticketing systems for railroads. There was a huge piece of electronics built by AT&T to process phone long distance billing records; all it really did was match call start and call end data on special paper tapes, then punch a card for each completed call. None of these were stored-program computers as we think of them today.

No big breakthroughs in this line of development yet; just incremental improvements.

The plugboards were a pain, and it was widely recognized that some better way to store programs and data would be a big help. Lots of things were tried - acoustic delay lines, drums, storage CRTs, magnetic core memory, plated wire memory... Magnetic cores were invented separately by several people, appearing about the same time in a British computer, an MIT computer, and a Seeburg jukebox. They were expensive, but worked.

IBM kept plugging away, producing the IBM 650, which was a programmable computer in the modern sense, but was mostly an upgrade path from the 604/609 series. Through the 1950s and early 1960s, IBM kept coming out with new and better models. There was a "business" line, with decimal arithmetic, and a "scientific" line, with binary arithmetic. Some of the programming arrangements were strange by modern standards; look up how the IBM 1401 did variable-length arithmetic with "word marks", how the 1620 had a decimal multiplication table in memory, and the strange addressing of the IBM 650.

Then IBM decided they had too many incompatible products, and developed the IBM System/360 family. One range of machines, all more or less compatible, with both binary and decimal arithmetic for both the scientific and business markets. Floating point, even. And a new way to make components - IBM Solid Logic Technology, individual transistors and other components placed into ceramic substrates by automated machinery. It wasn't quite an IC, but it was getting close. IBM now had something that looks pretty much like today's computers. Small and cheap were in the future, but the architecture had settled down. Binary arithmetic, byte-oriented, random-access memory, a reasonable instruction set, and a modest number of CPU registers had emerged as the winning architecture.

The early days were mostly about incremental improvement like that. Without Turing or Von Neumann, all this would have happened anyway.