Everyone should slow down AI development except for me • Hacker News Board

Re: Everyone should slow down AI development except for me

#171

Post by gpt5 » Sun, Sep 13, 2026, 5:10 AM UTC

ElProlactin wrote: post #161

hn_submit wrote: post #151

This is exactly the reason Anthropic's call to slow down A.I. development will fail. Shareholders will be loath to accept that the company loses its lead and a commensurate sharp decline in the company's share price. Moreover, it could lead China to catch up and eventually proclaim that it has nosed ahead of the U.S. in the field. The U.S. Government won't allow that to happen.

> The U.S. Government won't allow that to happen. Like the US government wouldn't allow Iran to close the Strait of Hormuz? Or a bunch of sandal-wearing Islamists to take control over the Red Sea coastline? The US government is not omnipotent. To the contrary, it's increasingly impotent. China has somewhere around 200x more shipbuilding capacity than the US today. It has more shipbuilding capacity in one shipyard tha…

You are right, US can do nothing on AI. After all, it doesn't have the shipbuilding capacity or even beat sandal-wearing islamists.

Re: Everyone should slow down AI development except for me

#172

Post by anon373839 » Sun, Sep 13, 2026, 5:12 AM UTC

rtpg wrote: post #163

lukan wrote: post #34

Earlier quoted context omitted.

Not sure about the level of irony here, but I keep hearing models have plateaued since a while now, but I keep being impressed with the latest model performance.

I'll take the opposite here. If someone put in frontier AI models from like .... last june I guess? in a box and let me run it with "decent" token throughput I would be happy. I think it's worth acknowledging that the power of LLMs at this point is not really so much in the smarts, but in the coordination and the surrounding harness tech. "Written english" turning into sequences of commands[0]. The whole agentic "stu…

> If someone put in frontier AI models from like .... last june I guess? in a box and let me run it with "decent" token throughput I would be happy.

You can have that! Qwen 3.8 Flash-Next is ~Opus 4.6 and runs nicely on a DGX Spark. And that’s just an architecture preview. The Qwen 4 family is expected to arrive this fall.

Re: Everyone should slow down AI development except for me

#173

Post by metalcrow » Sun, Sep 13, 2026, 5:24 AM UTC

fc417fc802 wrote: post #162

metalcrow wrote: post #148

Earlier quoted context omitted.

To further the metaphor, didn't the ban on nuclear weapons research work? We don't have pocket nukes, and there's very little risk of random countries acquiring their own due to the amount of work involved. To be concrete to AI: if we stop work on the frontier, the best we can do is make the current frontier easier to get to and more accessible. And while current frontier is strong, it's not world-changing so. The on…

Not in the way you seem to be thinking, no. Nuclear weapons can be banned because they are prohibitively expensive to research and build. You can't reliably hide a nuclear program. In contrast, rewind to the early 1800s and there is zero hope of a ban on the R&D of small arms being effective in the long run. The only thing it might maybe ensure is that no legitimate actors that fall under your jurisdiction are involv…

> Not in the way you seem to be thinking, no. Nuclear weapons can be banned because they are prohibitively expensive to research and build. You can't reliably hide a nuclear program.

Isn't that true of AI as well? Data centers are very big and very expensive.

> Bit of a tangent but we do, actually

Sure but not like the movies, these don't destroy cities. But the metaphor isn't accurate anyway: nukes don't get worse. The actual idea here is preventing the frontier of AI from advancing.

Re: Everyone should slow down AI development except for me

#174

Post by metalcrow » Sun, Sep 13, 2026, 5:26 AM UTC

defrost wrote: post #160

metalcrow wrote: post #148

Earlier quoted context omitted.

To further the metaphor, didn't the ban on nuclear weapons research work? We don't have pocket nukes, and there's very little risk of random countries acquiring their own due to the amount of work involved. To be concrete to AI: if we stop work on the frontier, the best we can do is make the current frontier easier to get to and more accessible. And while current frontier is strong, it's not world-changing so. The on…

> We don't have pocket nukes, What do you call the Davy Crockett warheads like the W54 ? The 0.3 kiloton low yield B61-12 ? The Chagai-I boosted fission warheads demonstrated by Pakistan in 1998 ? > and there's very little risk of random countries acquiring their own due to the amount of work involved. And yet North Korea, Isreal and Pakistan got there ... and India speed ran five tests in 1998 that caught the US com…

North Korea got it with help from China, Israel got it with help from France. They didn't do it on their own.

Re: Everyone should slow down AI development except for me

#175

Post by bedstefar » Sun, Sep 13, 2026, 5:27 AM UTC

lacy_tinpot wrote: post #170

AngryData wrote: post #150

Earlier quoted context omitted.

Why? There are numerous technologies we have ignored or abandoned for numerous reasons. Many of the benefits of AI is "requires less humans", but in an age where we question what work people could possibly do in the future, human labor isn't that hard to find. If the amish can do what they do for whack religious reasons, people could manage it with ai for cultural and social reasons. And I never saw an amish person s…

You can't just stop AI without also stopping progress in computer hardware. You have to do both. The real implication of what they're saying extends beyond just AI. You have to stop the entire compute stack in order to prevent or slow down AI progress.

And even if we did manage to limit/cull hardware development, what would keep people from continuing to find ever more efficient models runnable on contemporary hardware?

Re: Everyone should slow down AI development except for me

#176

Post by ares623 » Sun, Sep 13, 2026, 5:31 AM UTC

dvt wrote: post #165

Obligatory "what's Lygma?" But in all seriousness, this AI doomerism has reached a comedic inflection point. A few years ago, GPT-2 was too dangerous to release, then some Google weirdo said Gemini had a soul or something, then Mythos-tier became a meme, then some kid quit and went on FOX News talking about Skynet, and now Amodei and Altman are both on the "we need to slow down" train again; this is after we've alrea…

It all hinges on three letters: DJT

They just need to make enough waves, enough eye catching headlines, to make him look like a hero that swooped in to save the day.

Re: Everyone should slow down AI development except for me

#177

Post by ben_w » Sun, Sep 13, 2026, 5:31 AM UTC

Buttons840 wrote: post #104

AaronAPU wrote: post #27

They are talking about slowing down the public facing AI development. Because then nation states can create a capabilities gap between them and the public. Why does nobody seem to be pointing out this obvious explanation? It explains why the “we need to race China” concern suddenly vanished in the discussion. The government can simply gag Sam, Dario, Musk on national security basis, getting them all behind the public…

All the worst outcomes involve taking this technology away from the public.

The current state of alignment is that we don't know how to make it only as bad as Pol Pot was made by his parents, teachers, genetics, circumstances, etc.

This remains true regardless of if it is or isn't kept away from the public.

"Helpful, harness, and honest": when used by a bastard, even just helpful and harmless are in direct conflict with each other. you may hate the government, but what about every radical group of extremists that wants to take over your government? Are none of them worse?

None of this denies the problems with governments (if or not they take this tech for themselves and refuse it for others), just that it's a lack of imagination to say:

> All the worst outcomes involve taking this technology away from the public

Re: Everyone should slow down AI development except for me

#178

Post by ElProlactin » Sun, Sep 13, 2026, 5:34 AM UTC

gpt5 wrote: post #171

ElProlactin wrote: post #161

Earlier quoted context omitted.

> The U.S. Government won't allow that to happen. Like the US government wouldn't allow Iran to close the Strait of Hormuz? Or a bunch of sandal-wearing Islamists to take control over the Red Sea coastline? The US government is not omnipotent. To the contrary, it's increasingly impotent. China has somewhere around 200x more shipbuilding capacity than the US today. It has more shipbuilding capacity in one shipyard tha…

You are right, US can do nothing on AI. After all, it doesn't have the shipbuilding capacity or even beat sandal-wearing islamists.

Being sarcastic isn't going to solve the problem but you're obviously free to believe that the US is still in a position to call all the shots when it's evident to the rest of the world this isn't the case.

Re: Everyone should slow down AI development except for me

#179

Post by CamperBob2 » Sun, Sep 13, 2026, 5:38 AM UTC

dorolow wrote: post #46

nradov wrote: post #36

Earlier quoted context omitted.

Yikes. Hopefully you never gain any power or influence. The last thing we need is more economic central planning. I sincerely hope that you fail.

Do you think things like the Manhattan Project were a mistake? Comparing AI to nuclear weapons is perhaps a stretch, but I think most people recognize that certain technologies or artifacts are best monopolized by our governing bodies. I think if the capabilities of AI systems keep growing on trend, it is not unreasonable to think wonton usage could disrupt society or cause mass harm.

Trump and Hegseth cannot be compared to Roosevelt and Stimson. (Notwithstanding Hegseth's illegitimate attempt to rename the DoD to what it was in Stimson's day.)

Entirely different class of people altogether.

Re: Everyone should slow down AI development except for me

#180

Post by Retric » Sun, Sep 13, 2026, 5:41 AM UTC

fc417fc802 wrote: post #156

AngryData wrote: post #150

Earlier quoted context omitted.

Why? There are numerous technologies we have ignored or abandoned for numerous reasons. Many of the benefits of AI is "requires less humans", but in an age where we question what work people could possibly do in the future, human labor isn't that hard to find. If the amish can do what they do for whack religious reasons, people could manage it with ai for cultural and social reasons. And I never saw an amish person s…

But notably the amish don't preclude the rest of us from existing. Some subset of humans could turn their attention away from AI but it would presumably still exist and continue to be developed. I can't think of any economically beneficial technologies that we've collectively ignored. If you manage to come up with a counterexample then that's an opportunity to make some money for yourself. It's a fundamentally unstab…

Supersonic passenger aircraft were operated profitably and no longer exist.

There’s levels of R&D required for many technologies where the question goes beyond could this be profitable to what are the risk vs reward that this specific project will succeed.