The post-exponential era of AI and Moore’s Law

Which is dead wrong, because not only is there no usable definition of "improve itself", but there isn't even any understanding of the kinds of skills required to create a usable definition.

It's the difference between a computer that is taught how to compose okay-ish music, and a computer that learns spontaneously how to compose really really great music and do all of the social, cultural, and financial things required to create a career for itself as a notable composer and then does something entirely new and surprising given that starting point.

They're completely different problem classes, operating on completely different levels of sophistication and insight.

A lot of "real" AI problems are cultural, social, psychological, and semantic, and are going to need entirely new forms of meta-computation.

You're not going to get there with any current form of ML, no matter how fast it runs, because no current form of ML can represent the problems that need to be solved to operate in those domains - never mind spontaneously generate effective solutions for those problems.

A lot of the ways programmers write slow code ends up with multiplicative factors you can't practically recover with simply throwing more hardware at it, because it's really easy for us to write something 100x or 1000x slower (even in the same big-O class) by using a slow language with poor cache locality, that makes it easy to do more work than was necessary, by poorly using threading resources, etc. There's a lot of multiplicative factors that can stack together fairly quickly.

On the plus side, it turns out recovering most of that isn't that hard with a bit of careful language selection and just a bit of care in writing code. I think this is one of the somewhat subtle reasons for Go's success is that it trims away most of the slow things that a lot of "scripting" languages do without throwing away too much of the power, and I anticipate the continued entry and success of other languages like Nim into this space on the language chart in the next few years. To get to screaming fast may take a lot of effort, but "pretty fast, let's parallelize what we have instead of hyperoptimizing it" isn't too hard.

Because of all of these multiplicative slowdowns, even at very small scales it's still generally a better idea to start by optimizing and removing some of your multiplicative slowdowns before you just throw more hardware at it. Compute is merely cheap, not free. It's the rare bit of code that nobody's spent any time optimizing, but is also free of these multiplicative slowdown problems.

Hardware Acceleration/Parallization is the next frontier. We've already seen the benefit of some pretty simple ASICs (TPU was built to be simple) as well as more general purpose accelerators. Hardware architects used to have a hard time, because often the best option was to simply wait for CPUs to get faster. Now that we've seen CPU power begin to stall it makes economic sense not only to invest in more parallel software but more appliciation specific accelerators.

CPUs/GPUs are beasts of hardware architecture, being complex mostly due to their flexibility. We can achieve higher performance with dedicated hardware (or FPGAs), and it looks like the economic reasons to do so are slowly becoming more certain.

For a long time we had a situation where transistors got smaller and cheaper and faster and more power efficient all at the same through the wonders of Denard scaling and we gestured broadly at the whole thing and called it all "Moore's Law" without needing to distinguish which exponential improvement the term referred to. But in the mid 2000s Denard scaling broke down and now it looks like transistors are still getting smaller and cheaper and more power efficient but they aren't getting exponentially faster any more. So we've mostly settled on the smaller bit as being the true "Moore's Law" and that's more or less kept going but might be running out of steam. It's still very nice from the perspective of very parallel tasks but as consumers we don't see as much benefit from it in our computers except in graphics. But sooner or later Moore's Law will run out too, transistors can only get so small as long as they're made of atoms.

Luckily the laws of physics have precise limits for just how efficient computation can be, Landauer's principle, that are just as binding as the laws for how efficient a heat engine can be. Progress in engine efficiency did indeed form a converging sigmoid on what's allowed by Carnot. But we're still quite a distance away from Landauer's limit and if transistors can't get us there we have every reason to believe that other computational substrates are possible and can.

In the meantime we might be looking at an interregnum. But while AI is gobbling up computational cycles because they're available there's no reason to think that efficiency gains aren't possible - just look at the orders of magnitude improvement in training resources from AlphaGo to AlphaZero. I don't see that this should stop progress, though it might slow it.

[1]https://en.wikipedia.org/wiki/Landauer%27s_principle

> Which is dead wrong, because not only is there no usable definition of "improve itself", but there isn't even any understanding of the kinds of skills required to create a usable definition.

I disagree. A program improves itself when it reacts to a problem and implements a solution. Obviously that is very general, but enough. A human of IQ 100 certainly can develop software; a program of IQ 100 should be able to do the same, and then you scale horizontally.