Kurzweil's rebuttal to Paul Allen

This is an extremely common, and completely incorrect criticism as applied to AGI complexity estimates.

Here's the way to think of it: the entirety of the information content required to move from non-intelligence to intelligence has to have been figured out some time between single-celled organisms and humans, because the substrate on which our intelligence is implemented literally didn't exist at that point. Which means that any part of the biological machinery that was in place when single-celled organisms ruled the planet does not count towards the complexity of the "intelligence algorithm" itself - it's irrelevant, accidental complexity, not information content that is required to get from "working computer" to "working intelligent computer".

You'll find that almost all of the cellular actions, protein folding, and physics were already working just fine when the single-celled ickies were evolving, so it's all complexity that we can safely ignore, which means we can start the complexity count with DNA. Apart from (IMO) extremely minor epigenetic contributions, the pure-DNA information estimates should provide extremely hard upper bounds on the difficulty of the problem, estimates that we'll probably blow through quite easily once we know what we're doing - evolution rarely finds the optimal solutions to problems, I see no reason to assume that it stumbled across one in this case...

And yet nobody would ever claim that the fundamental minimum complexity of the "Hello, world!" algorithm was more than a few lines of text, because the "important stuff" in that algorithm has nothing to do with all of the irrelevant complexity in the operating system and hardware, all of which could be done in millions of different ways without changing the fundamental algorithmic insights that "Hello, World!" requires.

Yes, there are epigenetic factors that get passed along, but their effects tend to be transient, lost over a few generations at most, adding maybe a handful of tunable bits of information to the genome that can be quick-flipped as the environment demands. I don't know that anyone has ever suggested that any non-trivial amount of data is actually passed millions of generations down the line through this mechanism (keep in mind, to seriously take issue [i.e. beyond a mere factor of 2] with the estimates that you disagree with, you'd need to find more than 25 megabytes of evolutionarily-accessible data that lives somewhere other than DNA), and I'd be extremely interested to know if that was the case. Every evolutionary biologist I know focuses almost exclusively on genetic code as the evolutionary substrate because it's the only available channel that seems reliable enough for information to flow through unmolested over millions of years.

Would you really assume that the source code that it would take to write such a program would be over an order of magnitude different if you were targeting a RISC processor instead? Now give yourself access to an expressive compiled language, and estimate how much code it would take. Does the fact that you're targeting RISC even matter anymore, algorithmically?

Unless you're assuming that the biological "instruction set" has some hard coded primitives that make AI an easy problem, it literally doesn't matter at all that the instruction set is complicated (or rather, it matters up to a small constant factor), given that our programming constructs are vastly more powerful than those available to neurons. It's the connectivity algorithms that are important, and biology has absolutely no advantage there.

>It's true that if you look at most areas of technology they are advancing rapidly. Except energy. Energy has stagnated since the 1950s.

This is demonstrably false. Solar power generation, for example, has been enjoyinging the same kind of Moore's Law exponential price/power over the past 15 years that computer processing power has. This is hardly surprising due to the fact that silicon wafer solar panels often use the same semi-conductor suppliers that computer hardware manufacturers do. Newer thin-film solar panels represent a jump in paradigm that promises even greater price performance.

I could have brought up similar points about the progress of wind-power, bio-fuel, or a number of other fields. Energy has anything but stagnated since the 1950s.

tldr: Stay off the peak oil scaremongering sites. They'll blind you.

isn't this what Allen said in his critique? scientific achievement doesn't just grow exponentially - there are those "jumps in paradigm" that move us forward, but they're relatively rare and unpredictable.