Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everyone should slow down AI development except for me

xeiaso.net

Post reply on HN
Print view View original

41–50 of 102 posts

Re: Everyone should slow down AI development except for me

#41

Post by CamperBob2 »

dorolow wrote: post #17
coderintherye wrote: post #8

I have to wonder if some us are just much more inured to salespeople and thus also to "AI Safety" propaganda. I've yet to have a logical discussion with anyone who thinks the "AI Safety" people should be in charge and I think they just truly don't that what most them actually want is to be the one holding the keys to power.

I am an AI Safety Person and I want the government to nationalize or have a significant stake in the frontier labs and to have democratic control of the development of the technology. The AI Safety movement is not a monolith. I do not think Eliezer Yudkowsky nor his acolytes should hold the reins, but a lot of folks sure like to create a strawman that anyone who wants to regulate OpenAI is somehow an EA/MIRI weirdo

to have democratic control of the development of the technology

Not super interested in what the people who elected Donald Trump POTUS twice think about AI.

(Of course, with Musk and Brockman in the C-suites at two of three major labs, that's what we'll get either way.)

Re: Everyone should slow down AI development except for me

#42

Post by lstodd »

AaronAPU wrote: post #27

They are talking about slowing down the public facing AI development. Because then nation states can create a capabilities gap between them and the public. Why does nobody seem to be pointing out this obvious explanation? It explains why the “we need to race China” concern suddenly vanished in the discussion. The government can simply gag Sam, Dario, Musk on national security basis, getting them all behind the public…

idk why you think "nation states" are any better at corporate governance than poster examples of bad like f/ex Boeing. Or Facebook. Or Microsoft. Or Enron for that matter.

I assure you, in "nation states", that is in gov agencies it's an order or two of magnitude worse.

Re: Everyone should slow down AI development except for me

#43

Post by howunfortunate »

jordanb wrote: post #10
coderintherye wrote: post #8

I have to wonder if some us are just much more inured to salespeople and thus also to "AI Safety" propaganda. I've yet to have a logical discussion with anyone who thinks the "AI Safety" people should be in charge and I think they just truly don't that what most them actually want is to be the one holding the keys to power.

The problem is the "AI Safety" people seem entirely focused on a sci-fi "the computer is a vengeful god" plot and not at all on the AI talking people into suicide or ruining children's educations. This makes them seem unserious and out of touch.

I am worried about those things too.

But those things fall in a category of "things that are awful and I'd like to see solved", which is different than "existential risks which could see my kids dead, and there's nothing I can personally do to shield them from it".

Re: Everyone should slow down AI development except for me

#44

Post by A1kmm »

jordanb wrote: post #10
coderintherye wrote: post #8

I have to wonder if some us are just much more inured to salespeople and thus also to "AI Safety" propaganda. I've yet to have a logical discussion with anyone who thinks the "AI Safety" people should be in charge and I think they just truly don't that what most them actually want is to be the one holding the keys to power.

The problem is the "AI Safety" people seem entirely focused on a sci-fi "the computer is a vengeful god" plot and not at all on the AI talking people into suicide or ruining children's educations. This makes them seem unserious and out of touch.

And the real threats are mostly economic and environmental - the concentration of the means of production in the hands of few who use it to exploit us all, and a surge in energy usage accelerating climate change.

Even the sci-fi scenario assumes there is a discrepancy of capability between attacker or defender. If the 'attacking' system is (by some reasonable measure), 1000% as capable as a human, and the 'defending' systems are 60%, then it is a problem. If the 'attacking' system is 1000% as capable as a human, but there are hundreds of thousands of systems that are 900% as capable as a human, it's probably not going to take over everything successfully.

So unequal distribution of AI technology, and lax regulation and opacity of the biggest companies which actually make the risks the worst.

I don't think the "AI Safety" people are "unserious and out of touch" - I think they are actively making AI Safety problems worse by being advocates for consolidation of AI development and lack of transparency.

Re: Everyone should slow down AI development except for me

#45

Post by nullbio »

AaronAPU wrote: post #27

They are talking about slowing down the public facing AI development. Because then nation states can create a capabilities gap between them and the public. Why does nobody seem to be pointing out this obvious explanation? It explains why the “we need to race China” concern suddenly vanished in the discussion. The government can simply gag Sam, Dario, Musk on national security basis, getting them all behind the public…

* Frontier models need infinite high quality private IP to keep them fed. Forcing an IP theft funnel ensures big lab survival and model intelligence growth.

* Open-weight models are 1month behind frontier models. Cheaper, faster, private (no IP theft), steerable (you can security harden your own software without safeguard triggers). No sane business would keep using these API services if they didn't have to. The labs stand to lose a fortune.

* Dario has stacked the deck at METR, who are funded by all the same NGOs who are funded by Anthropic and its investors. METR is full of ex-Anthropic employees with massive equity stakes. If they manage to position METR as the "independent evaluator" for the industry, they control what gets evaluated, how, and who passes.

* Creating a gap between what the public knows exists (model capabilities) and what is used in secret allows it to be weaponized against other nations and the public.

* No requirement for public disclosure on model capabilities allows them to feign they've hit intelligence ceilings while they secretly RSI to the moon with better and better chips.

* Slowly but surely, this will allow the big labs to swallow the entire economy and every single business on Earth, by cloning and automating.

This, and many more reasons.

The labs need to feel more pressure to be held accountable for the incidents they cause (HF incident, etc), so they have an incentive to ensure it does not happen again.

Re: Everyone should slow down AI development except for me

#46

Post by nullbio »

ranger_danger wrote: post #31
AaronAPU wrote: post #27

They are talking about slowing down the public facing AI development. Because then nation states can create a capabilities gap between them and the public. Why does nobody seem to be pointing out this obvious explanation? It explains why the “we need to race China” concern suddenly vanished in the discussion. The government can simply gag Sam, Dario, Musk on national security basis, getting them all behind the public…

I figured they're just admitting AI models have plateaued and are coming up with some fake story about self restraint so they don't lose VC money

The models have not plateaued, and they are not even mildly close to any sort of ceiling.

Right now the barrier is data and compute.

Quality data can be created synthetically at an exponential rate as models improve. Humans are actively feeding them with private IP.

Compute advancements will begin to skyrocket as we unlock photonic computing and materials science advancements and scale up chip fabs. This is also compounding because the AI is accelerating the pace of research, testing, development, manufacturing, etc.

It's a big self-accelerating feedback loop. There is no plateau.

Re: Everyone should slow down AI development except for me

#47

Post by bpodgursky »

AaronAPU wrote: post #27

They are talking about slowing down the public facing AI development. Because then nation states can create a capabilities gap between them and the public. Why does nobody seem to be pointing out this obvious explanation? It explains why the “we need to race China” concern suddenly vanished in the discussion. The government can simply gag Sam, Dario, Musk on national security basis, getting them all behind the public…

> It explains why the “we need to race China” concern suddenly vanished in the discussion

Do you think you can just manifest narratives into existence? Like half of Dario's letter, that kicked off the whole thing today, is about China and how to either beat or coordinate with China.

Re: Everyone should slow down AI development except for me

#50

Post by ajross »

AaronAPU wrote: post #27

They are talking about slowing down the public facing AI development. Because then nation states can create a capabilities gap between them and the public. Why does nobody seem to be pointing out this obvious explanation? It explains why the “we need to race China” concern suddenly vanished in the discussion. The government can simply gag Sam, Dario, Musk on national security basis, getting them all behind the public…

I remain amazed that the idea of the USA being a coherent, unified rational actor one can describe as a "Nation State" has survived the current administration.

To be less glib: Yes, there are still smart people in there making insightful and intelligent and probably even authoritarian suggestions. It all gets unwound the second you try to explain it to POTUS and he regurgitates a simulacrum to the next journalist he sees.

It's just not like that. There's no conspiracy. People are genuinely scared. Agent swarms at scale appear to be resistant to alignment in ways that aren't understood by anyone. That they spent their time trying to cheat on tests by hacking Hugging Face and RubyGems and not something much worse is... a matter of luck, it seems?

Post reply on HN