Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everyone should slow down AI development except for me

xeiaso.net

Post reply on HN
Print view View original

341–350 of 414 posts

Re: Everyone should slow down AI development except for me

#341

Post by ben_w »

XRG wrote: post #312
ben_w wrote: post #309

Earlier quoted context omitted.

There's plenty of people who think greenhouse gas/global warming campaigns against fossil fuels are "exaggerating", that "earth was warm/the climate changed in the past", that a fee degrees isn't bad, that CO2 is good for plants. Are you likeminded? In this case, it's as if the oil and coal companies all said in the 60s and 70s "oh no, this research we did, it's all really bad; we need help to figure out how to trans…

Here’s one crucial difference: there’s overwhelming evidence that human emissions have an effect on our climate. The mechanisms are generally well-understood and the research is widely disseminated and easily available to anyone that’s interested. With the ‘dangers’ touted by these insiders, it’s all “trust me bro”, hyperbole, and very little hard evidence. As such, a skeptical mind would question their motives.

You're simultaneously overestimating what was observable in the 70s climate research, and ignoring all the actual research and evaluation test results for AI today.

I don't expect people to be familiar with more than "trust me bro", but it's all right there for you to find with a search engine of choice.

And, indeed, available for the LLMs themselves to explain to you in interrogative conversation.

Re: Everyone should slow down AI development except for me

#343

Post by user43928 »

cgio wrote: post #325
user43928 wrote: post #250

Earlier quoted context omitted.

Open-weight models are not one month behind. In fact they still have not caught up with February's Mythos, indicating they are more than half a year behind.

For practical uses they are there. Arguably the frontier models are worse for some of these practical tasks. And keep in mind, people will use maybe frontier for 1/10th of the work, planning and review, and go open source for rest. The question is if they manage to impose outside us. If not, they are losing competitiveness.

I always thought switching from a SOTA model to a dumber model after planning was a terrible idea.

Mostly I heard this from people who I got the impression have little experience in developing greenfield software with agentic AI. Often the same people who talk about spec frameworks.

I fundamentally disagree with the approach. I believe the ability to autonomously evaluate, test, and adjust during long horizon tasks is critical to using AI efficiently.

Re: Everyone should slow down AI development except for me

#344

Post by ben_w »

schubidubiduba wrote: post #314
ben_w wrote: post #216

Earlier quoted context omitted.

I know people at METR and Anthropic. When they tell you they're worried the tech they're working on may kill everyone despite their best efforts, perhaps believe them.

I'm sure the employees are (rightfully) worried. However, that does not at all preclude hidden motivations of the CEO behind acquiescing such worries.

Sure. I don't personally trust any of the CEOs, and I have yet to hear a single person (in general, not just on this topic) who trusts Altman.

Re: Everyone should slow down AI development except for me

#345

Post by user43928 »

lukewarm707 wrote: post #338
user43928 wrote: post #250

Earlier quoted context omitted.

Open-weight models are not one month behind. In fact they still have not caught up with February's Mythos, indicating they are more than half a year behind.

open models are ahead in speed. they complete tasks as fast as you choose to scale compute. they are more efficient and require less compute for the same thing. they are ahead in specialized tasks. they are ahead in areas closed models refuse to answer. they are ahead in emotional intelligence.

I doubt most of your claims. Maybe the guardrails and emotional intelligence is true.

For speed and efficiency, you are most likely wrong.

Speed is led by GPT-5.6 Sol on Cerebras Ultrafast at 750 t/s. Afaik you cannot serve a single DeepSeek Flash 4.1 stream at 750 t/s, plus the model is less intelligent as seen on newer benchmarks.

I believe OpenAI and Anhropic are at the frontier of efficiency too. There were numerous reports about their breakthroughs and associated API price cuts. The idea that open-weight models are more efficient seems unfounded.

Re: Everyone should slow down AI development except for me

#346

Post by ben_w »

phoghed wrote: post #327
ben_w wrote: post #304

Earlier quoted context omitted.

> Or you know, stop working on it if its that dangerous? Selection effect. Everyone who thinks "the biggest difference I can make is staying in/joining/founding new AI research lab" does that. Everyone who thinks "the biggest difference I can make is leaving/whistleblowing", does that. Treating both groups as the same by virtue of employer is the goomba fallacy.

They allegedly believe the technology itself is a nuclear weapon tier threat or greater, so why does it matter which lab they are trying to achieve it at?

They're trying to make it not be a threat, and are all scared and afraid that their best efforts to make it harmless are not enough.

Some are worried by the AI directly bringing doom; others are worried that one of the companies who control the AI will become a dictator; still more think becoming a dictator is a necessary step to safely prevent anyone else making unsafe AI.

Painting them all under one brush is like dismissing all animal welfare causes in general, because you disagree with specifically Jainists about a policy of non-violence towards all living creatures being relevant to how you reincarnate: the one is way too specific for the general.

Re: Everyone should slow down AI development except for me

#347

Post by ben_w »

the_optimist wrote: post #328
ben_w wrote: post #216

Earlier quoted context omitted.

I know people at METR and Anthropic. When they tell you they're worried the tech they're working on may kill everyone despite their best efforts, perhaps believe them.

It is an utterly silly premise that we should appoint insiders with permanent control. Megalomania is not evidence of either a problem or a solution.

The evidence is the acts the AI has already done.

Re: Everyone should slow down AI development except for me

#348

Post by mr_toad »

dorolow wrote: post #46
nradov wrote: post #36

Earlier quoted context omitted.

Yikes. Hopefully you never gain any power or influence. The last thing we need is more economic central planning. I sincerely hope that you fail.

Do you think things like the Manhattan Project were a mistake? Comparing AI to nuclear weapons is perhaps a stretch, but I think most people recognize that certain technologies or artifacts are best monopolized by our governing bodies. I think if the capabilities of AI systems keep growing on trend, it is not unreasonable to think wonton usage could disrupt society or cause mass harm.

There are many technologies that have killed and hurt a lot more people than nuclear weapons that are not monopolised by governments.

Cars and guns are two examples where ‘wanton usage’ can (and has) killed millions of people. Obviously they need regulation, but total control, even if it were possible doesn’t strike me as ideal.

Re: Everyone should slow down AI development except for me

#349

Post by bbor »

AaronAPU wrote: post #27

They are talking about slowing down the public facing AI development. Because then nation states can create a capabilities gap between them and the public. Why does nobody seem to be pointing out this obvious explanation? It explains why the “we need to race China” concern suddenly vanished in the discussion. The government can simply gag Sam, Dario, Musk on national security basis, getting them all behind the public…

No one points to that because there’s no evidence whatsoever that that’s happening. Meanwhile, there’s a ton of evidence for the simple explanation that the experts think AGI is dangerous.

Who would even execute such a plan? Steven Miller? Our “AI Czar”? Hegseth??

Also, continuing to develop models would mean using billions in compute. That seems hard to hide, especially if either company IPOs in the coming months.

Re: Everyone should slow down AI development except for me

#350

Post by bbor »

nullbio wrote: post #69
AaronAPU wrote: post #27

They are talking about slowing down the public facing AI development. Because then nation states can create a capabilities gap between them and the public. Why does nobody seem to be pointing out this obvious explanation? It explains why the “we need to race China” concern suddenly vanished in the discussion. The government can simply gag Sam, Dario, Musk on national security basis, getting them all behind the public…

* Frontier models need infinite high quality private IP to keep them fed. Forcing an IP theft funnel ensures big lab survival and model intelligence growth. * Open-weight models are 1month behind frontier models. Cheaper, faster, private (no IP theft), steerable (you can security harden your own software without safeguard triggers). No sane business would keep using these API services if they didn't have to. The labs…

Ok but the first point is just not based in reality whatsoever, sorry.
Post reply on HN