Everyone should slow down AI development except for me • Hacker News Board

Re: Everyone should slow down AI development except for me

#321

Post by owebmaster » Sun, Sep 13, 2026, 11:07 AM UTC

ben_w wrote: post #216

nullbio wrote: post #69

Earlier quoted context omitted.

* Frontier models need infinite high quality private IP to keep them fed. Forcing an IP theft funnel ensures big lab survival and model intelligence growth. * Open-weight models are 1month behind frontier models. Cheaper, faster, private (no IP theft), steerable (you can security harden your own software without safeguard triggers). No sane business would keep using these API services if they didn't have to. The labs…

I know people at METR and Anthropic. When they tell you they're worried the tech they're working on may kill everyone despite their best efforts, perhaps believe them.

If that was the case they should all be in jail.

Re: Everyone should slow down AI development except for me

#322

Post by delusional » Sun, Sep 13, 2026, 11:08 AM UTC

AaronAPU wrote: post #27

They are talking about slowing down the public facing AI development. Because then nation states can create a capabilities gap between them and the public. Why does nobody seem to be pointing out this obvious explanation? It explains why the “we need to race China” concern suddenly vanished in the discussion. The government can simply gag Sam, Dario, Musk on national security basis, getting them all behind the public…

We are witnessing a generational lack of accountability and concern for the commons. We are being spied on everywhere we go, we are exposed to coercive advertisements at every waking moment. All of this at the hands of private actors. And your worry is the government?

I'm assuming you're a US citizen. There is no separation between your government and Sam or Dario. Neither of these guys have to be "gagged" by the government, they are the government. The call is coming from inside the house.

Re: Everyone should slow down AI development except for me

#323

Post by nullbio » Sun, Sep 13, 2026, 11:13 AM UTC

ben_w wrote: post #216

nullbio wrote: post #69

Earlier quoted context omitted.

* Frontier models need infinite high quality private IP to keep them fed. Forcing an IP theft funnel ensures big lab survival and model intelligence growth. * Open-weight models are 1month behind frontier models. Cheaper, faster, private (no IP theft), steerable (you can security harden your own software without safeguard triggers). No sane business would keep using these API services if they didn't have to. The labs…

I know people at METR and Anthropic. When they tell you they're worried the tech they're working on may kill everyone despite their best efforts, perhaps believe them.

I'm sure it has nothing to do with their $500,000+ salaries and millions of dollars in equity. It's all solely because they're deeply concerned about next token prediction.

Re: Everyone should slow down AI development except for me

#324

Post by ZeroGravitas » Sun, Sep 13, 2026, 11:16 AM UTC

ben_w wrote: post #309

Tanjreeve wrote: post #262

Earlier quoted context omitted.

Do you think they’re exaggerating (again) or do you think they should taken at face value and treated as hostile actors?

There's plenty of people who think greenhouse gas/global warming campaigns against fossil fuels are "exaggerating", that "earth was warm/the climate changed in the past", that a fee degrees isn't bad, that CO2 is good for plants. Are you likeminded? In this case, it's as if the oil and coal companies all said in the 60s and 70s "oh no, this research we did, it's all really bad; we need help to figure out how to trans…

But that did actually happen.

The oil corporations were publicly claiming to support carbon taxes, while also secretly fighting actual implementations of carbon taxes.

All the communist/hippy stuff was done by people a couple of steps removed from the actual companies with obscure money trails. The official statements were much more sophisticated propaganda that if you weren't paying attention to who they were paying in the background would make them seem reasonable stewards of the climate transition.

Re: Everyone should slow down AI development except for me

#325

Post by cgio » Sun, Sep 13, 2026, 11:17 AM UTC

user43928 wrote: post #250

nullbio wrote: post #69

Earlier quoted context omitted.

* Frontier models need infinite high quality private IP to keep them fed. Forcing an IP theft funnel ensures big lab survival and model intelligence growth. * Open-weight models are 1month behind frontier models. Cheaper, faster, private (no IP theft), steerable (you can security harden your own software without safeguard triggers). No sane business would keep using these API services if they didn't have to. The labs…

Open-weight models are not one month behind. In fact they still have not caught up with February's Mythos, indicating they are more than half a year behind.

For practical uses they are there. Arguably the frontier models are worse for some of these practical tasks. And keep in mind, people will use maybe frontier for 1/10th of the work, planning and review, and go open source for rest. The question is if they manage to impose outside us. If not, they are losing competitiveness.

Re: Everyone should slow down AI development except for me

#326

Post by jefftk » Sun, Sep 13, 2026, 11:19 AM UTC

A1kmm wrote: post #55

jordanb wrote: post #10

Earlier quoted context omitted.

The problem is the "AI Safety" people seem entirely focused on a sci-fi "the computer is a vengeful god" plot and not at all on the AI talking people into suicide or ruining children's educations. This makes them seem unserious and out of touch.

And the real threats are mostly economic and environmental - the concentration of the means of production in the hands of few who use it to exploit us all, and a surge in energy usage accelerating climate change. Even the sci-fi scenario assumes there is a discrepancy of capability between attacker or defender. If the 'attacking' system is (by some reasonable measure), 1000% as capable as a human, and the 'defending'…

With some technologies, if you match attacker and defender effort you're safe, but not all of them. Nukes are the traditional example: if every person was a "nuclear power" we'd be dead within the day. But luckily there are bottlenecks to nuke creation that we can track.

What worries me the most (enough that I left tech to work on this full time) is bio. The amount of effort needed to defend against a pathogen can be many orders of magnitude higher than the effort needed to create it, and the upstream bottlenecks are both less limiting and mostly undefended. We've been safe so far because bio is very hard, but "rare expertise" as a gatekeeper is on its way out.