AI 2040: Plan A

It's not clear in this context what you actually mean by "government." You are assigning agency to something in a way that seems like a reification. While a bureaucracy can seem to have a life of its own, isn't it generally people who seek power?

America is not suffering from too many golf courses being constructed. They are, rather, going extinct, and I don't really think the mass loss of green spaces and third spaces is necessarily a good thing, even though I'm not someone who enjoys playing golf and don't really spend any time at golf courses.

> AI labs may also be able to reap tremendous benefit from these inference-scaled models by using them as part of the training process. If so, the large scale-up of compute resources could go into post-training rather than deployment. This would have very different implications for AI governance.

> ...

> So iterated distillation and amplification provides a plausible pathway for scaling inference-during-training to rapidly create much more powerful AI systems. Arguably this would constitute a form of ‘recursive self-improvement’ where AI systems are applied to the task of improving their own capabilities, leading to a rapid escalation.

So "inference scaling is required to scale capabilities" doesn't mean that we're reaching the top of the S-curve in intelligence. If anything, it could mean a shorter timeline and more unpredictable landscape for governance (e.g. due to securing weights no longer as effectively preventing escalation, more in the article).

There seems to be an alarmingly high amount of questionable data centre construction going on, such as projects being built in places with no access to power with an assumption they can somehow force the utility to provide it later. These buildouts seem to be being done for financial reasons (they are not Meta, Amazon, Azure, etc. facilities) with the hope to lease them out or sell them half-completed in the future. People rightfully don't want that kind of thing in their back yard.

To give a feel for the scale involved, this one (the new Amazon east DC) in my podunk area of the state is 250 MW (the existing us-east-2 in Columbus is 200 MW, although I'm not clear if this will be a new region or is just an additional availability zone). But that's small potatoes compared to the speculative project in Piketon, which amongst more absurd things is planned to be:

- 10 gigawatts (equal to 50% of current power consumption statewide) - "Modular" nuclear reactors built on site - 35,000 construction workers needed to build it (in a county with a total population less than that) - $30-$40 billion for the data centre, plus another $33 billion to build the 9 gigawatt natural gas electric plant - Meta agreeing to build an additional 1.2 GW nuclear plant on site - OpenAI in negotiations to lease the facility

This is a really big project, of the scale of "nothing like this has ever been done before". Nobody has ever built that much power generation at a single site before, nor has a datacentre this large ever been constructed. There is a very real risk of the project getting halfway done and then being unable to be completed. The prospect of a state literally doubling its electric generation is a bit ambitious, too (doing such means basically a complete revamp of the power distribution grid, or else some very novel designs to only use the power locally). For example, the normal type of shutoffs data centres have to prevent eg an incoming have are unacceptable in this situation because the grid cannot cope with 20 GW of demand suddenly disappearing.