Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everyone should slow down AI development except for me

xeiaso.net

Post reply on HN
Print view View original

321–330 of 335 posts

Re: Everyone should slow down AI development except for me

#321

Post by owebmaster »

ben_w wrote: post #216
nullbio wrote: post #69

Earlier quoted context omitted.

* Frontier models need infinite high quality private IP to keep them fed. Forcing an IP theft funnel ensures big lab survival and model intelligence growth. * Open-weight models are 1month behind frontier models. Cheaper, faster, private (no IP theft), steerable (you can security harden your own software without safeguard triggers). No sane business would keep using these API services if they didn't have to. The labs…

I know people at METR and Anthropic. When they tell you they're worried the tech they're working on may kill everyone despite their best efforts, perhaps believe them.

If that was the case they should all be in jail.

Re: Everyone should slow down AI development except for me

#322

Post by delusional »

AaronAPU wrote: post #27

They are talking about slowing down the public facing AI development. Because then nation states can create a capabilities gap between them and the public. Why does nobody seem to be pointing out this obvious explanation? It explains why the “we need to race China” concern suddenly vanished in the discussion. The government can simply gag Sam, Dario, Musk on national security basis, getting them all behind the public…

We are witnessing a generational lack of accountability and concern for the commons. We are being spied on everywhere we go, we are exposed to coercive advertisements at every waking moment. All of this at the hands of private actors. And your worry is the government?

I'm assuming you're a US citizen. There is no separation between your government and Sam or Dario. Neither of these guys have to be "gagged" by the government, they are the government. The call is coming from inside the house.

Re: Everyone should slow down AI development except for me

#323

Post by nullbio »

ben_w wrote: post #216
nullbio wrote: post #69

Earlier quoted context omitted.

* Frontier models need infinite high quality private IP to keep them fed. Forcing an IP theft funnel ensures big lab survival and model intelligence growth. * Open-weight models are 1month behind frontier models. Cheaper, faster, private (no IP theft), steerable (you can security harden your own software without safeguard triggers). No sane business would keep using these API services if they didn't have to. The labs…

I know people at METR and Anthropic. When they tell you they're worried the tech they're working on may kill everyone despite their best efforts, perhaps believe them.

I'm sure it has nothing to do with their $500,000+ salaries and millions of dollars in equity. It's all solely because they're deeply concerned about next token prediction.

Re: Everyone should slow down AI development except for me

#324

Post by ZeroGravitas »

ben_w wrote: post #309
Tanjreeve wrote: post #262

Earlier quoted context omitted.

Do you think they’re exaggerating (again) or do you think they should taken at face value and treated as hostile actors?

There's plenty of people who think greenhouse gas/global warming campaigns against fossil fuels are "exaggerating", that "earth was warm/the climate changed in the past", that a fee degrees isn't bad, that CO2 is good for plants. Are you likeminded? In this case, it's as if the oil and coal companies all said in the 60s and 70s "oh no, this research we did, it's all really bad; we need help to figure out how to trans…

But that did actually happen.

The oil corporations were publicly claiming to support carbon taxes, while also secretly fighting actual implementations of carbon taxes.

All the communist/hippy stuff was done by people a couple of steps removed from the actual companies with obscure money trails. The official statements were much more sophisticated propaganda that if you weren't paying attention to who they were paying in the background would make them seem reasonable stewards of the climate transition.

Re: Everyone should slow down AI development except for me

#325

Post by cgio »

user43928 wrote: post #250
nullbio wrote: post #69

Earlier quoted context omitted.

* Frontier models need infinite high quality private IP to keep them fed. Forcing an IP theft funnel ensures big lab survival and model intelligence growth. * Open-weight models are 1month behind frontier models. Cheaper, faster, private (no IP theft), steerable (you can security harden your own software without safeguard triggers). No sane business would keep using these API services if they didn't have to. The labs…

Open-weight models are not one month behind. In fact they still have not caught up with February's Mythos, indicating they are more than half a year behind.

For practical uses they are there. Arguably the frontier models are worse for some of these practical tasks. And keep in mind, people will use maybe frontier for 1/10th of the work, planning and review, and go open source for rest. The question is if they manage to impose outside us. If not, they are losing competitiveness.

Re: Everyone should slow down AI development except for me

#326

Post by jefftk »

A1kmm wrote: post #55
jordanb wrote: post #10

Earlier quoted context omitted.

The problem is the "AI Safety" people seem entirely focused on a sci-fi "the computer is a vengeful god" plot and not at all on the AI talking people into suicide or ruining children's educations. This makes them seem unserious and out of touch.

And the real threats are mostly economic and environmental - the concentration of the means of production in the hands of few who use it to exploit us all, and a surge in energy usage accelerating climate change. Even the sci-fi scenario assumes there is a discrepancy of capability between attacker or defender. If the 'attacking' system is (by some reasonable measure), 1000% as capable as a human, and the 'defending'…

With some technologies, if you match attacker and defender effort you're safe, but not all of them. Nukes are the traditional example: if every person was a "nuclear power" we'd be dead within the day. But luckily there are bottlenecks to nuke creation that we can track.

What worries me the most (enough that I left tech to work on this full time) is bio. The amount of effort needed to defend against a pathogen can be many orders of magnitude higher than the effort needed to create it, and the upstream bottlenecks are both less limiting and mostly undefended. We've been safe so far because bio is very hard, but "rare expertise" as a gatekeeper is on its way out.

Re: Everyone should slow down AI development except for me

#327

Post by phoghed »

ben_w wrote: post #304
calgoo wrote: post #258

Earlier quoted context omitted.

Or you know, stop working on it if its that dangerous? This whole thing of a bunch of employees saying that they are scared of building what they are building, but do it anyway because they are somehow going to make it different? Their model has been used in the planning of mass murdering in war as well as spying on the entire worlds population as well as helping ICE out in the US. They need to stop this BS fearmonge…

> Or you know, stop working on it if its that dangerous? Selection effect. Everyone who thinks "the biggest difference I can make is staying in/joining/founding new AI research lab" does that. Everyone who thinks "the biggest difference I can make is leaving/whistleblowing", does that. Treating both groups as the same by virtue of employer is the goomba fallacy.

They allegedly believe the technology itself is a nuclear weapon tier threat or greater, so why does it matter which lab they are trying to achieve it at?

Re: Everyone should slow down AI development except for me

#328

Post by the_optimist »

ben_w wrote: post #216
nullbio wrote: post #69

Earlier quoted context omitted.

* Frontier models need infinite high quality private IP to keep them fed. Forcing an IP theft funnel ensures big lab survival and model intelligence growth. * Open-weight models are 1month behind frontier models. Cheaper, faster, private (no IP theft), steerable (you can security harden your own software without safeguard triggers). No sane business would keep using these API services if they didn't have to. The labs…

I know people at METR and Anthropic. When they tell you they're worried the tech they're working on may kill everyone despite their best efforts, perhaps believe them.

It is an utterly silly premise that we should appoint insiders with permanent control.

Megalomania is not evidence of either a problem or a solution.

Re: Everyone should slow down AI development except for me

#329

Post by the_optimist »

int_19h wrote: post #268
calgoo wrote: post #258

Earlier quoted context omitted.

Or you know, stop working on it if its that dangerous? This whole thing of a bunch of employees saying that they are scared of building what they are building, but do it anyway because they are somehow going to make it different? Their model has been used in the planning of mass murdering in war as well as spying on the entire worlds population as well as helping ICE out in the US. They need to stop this BS fearmonge…

Anthropic specifically is basically saying that they believe it's even more dangerous if someone else gets to AGI before they do, so they have to either stop everyone or not stop themselves. I personally disagree with that take - and, as you note, it's hard to take seriously ethical wrangles from a company that literally sued the government in court to allow their models to be used by Palantir of all people. But if o…

It’s a bit of a self-serving argument, don’t you think?

And how does that relate to the ask for oligopoly licensing within global democracy?

“We must build the nuclear bomb first in order to make sure no one else builds one.” This the most nonsense, disingenuous argument imaginable.

Post reply on HN