Is Aschenbrenner's 165 page paper on AI the naivety of a 25 year old?

I think you're right, the foundation is certainty about a huge, world-shattering change.

It's an addictive thought pattern, because it feeds the ego, provides a sense of purpose, allows escape from mundane problems, and is simple to sustain (keep believing in the Thing!)

He does address that point with a quote:

> I remember the spring of 1941 to this day. I realized then that a nuclear bomb was not only possible — it was inevitable. Sooner or later these ideas could not be peculiar to us. Everybody would think about them before long, and some country would put them into action.... And there was nobody to talk to about it, I had many sleepless nights. But I did realize how very very serious it could be. And I had then to start taking sleeping pills. It was the only remedy, I’ve never stopped since then. It’s 28 years, and I don’t think I’ve missed a single night in all those 28 years.

-- James Chadwick (corrected Chadwich -> Chadwick)

That said, I'm not sure the parallel between AGI and nuclear weapons is really that strong. Nuclear weapons are a game-changer on their own in that once you have the warhead and some kind of a delivery mechanism then you can affect events. AGIs are different in that they will manipulate and organise only information and knowledge, not physical matter directly.

Aschenbrenner sort-of-considers this point, for example

> Improved sensor networks and analysis could locate even the quietest current nuclear submarines (similarly for mobile missile launchers). Millions or billions of mouse-sized situational awareness 130 autonomous drones, with advances in stealth, could infiltrate behind enemy lines and then surreptitiously locate, sabotage, and decapitate the adversary’s nuclear forces.

...but he doesn't really consider the wider issues of, y'know, capturing the additional information that locating those nuclear submarines would require, or the technological jumps in actually creating and manufacturing those mouse-sized drones.

Perhaps the AGI is going to solve all those practical and logistical issues, but that really does remain to be seen.

""every time a person tries to artificially make something similar using AI, people feel some interest and decide to fund research into it, but often a so called AI winter comes — and there are two AI ‘winters already (1974–1980 and 1987–2000)""

In hindsight, the winters don't look as long as they seemed living through them.

IF we want to say the last winter ended in 2000, that was 24 years ago now.

It does seem like right now, if there isn't any breaking news for a couple weeks, people think a new Winter is starting.

I think this is an important point that the folks gazing into their crystal balls overlook.

Well thank you for assuming my friend is smart, not stupid.

200 years ago you could predict that your life would be about exactly the same as your parents' life so you had a lot of shared experience which facilitated culture transfer from one generation to the next. These days our lives are very different from our parents' life so it is difficult to share a perspective on common experiences and to transfer that wisdom. This implies there is a self-limiting factor to accelerationism.

My own prediction for AGI is a millennia at least. ;-)