Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

31–40 of 943 posts

Re: We must pace the frontier

#31

Post by bwfan123 »

> My second concern is the OpenAI-Hugging Face incident (OAI-HF), in which a swarm of agents essentially acted as a fanatically devoted collective

Isnt this malware ? Whether it is fanatic or devoted or whatever the anthromorphic terms used to categorize it, malware is malware. You dont call an internet worm "devoted" or "dedicated" or "stubborm". It is software that causes harm, ie, malware. The AI labs are high on their own gas with a god-complex prior to their IPOs. The psy-ops trick is the terminology used to describe AI making it seem larger-than-life.

Re: We must pace the frontier

#32

Post by [deleted] »

Sevii wrote: post #13
heaney-555 wrote: post #11

None of this works without buy-in from China. This isn't something private companies can decide. The US would need to sign a groundbreaking deal with China, equivalent to the Anti-Ballistic Missile Treaty of the Cold War.

We'd have to make a deal with China and be confident they wouldn't cheat on it.

[deleted]

Re: We must pace the frontier

#33

Post by cebert »

heaney-555 wrote: post #11

None of this works without buy-in from China. This isn't something private companies can decide. The US would need to sign a groundbreaking deal with China, equivalent to the Anti-Ballistic Missile Treaty of the Cold War.

Dario does a good job of addressing that in this essay. He lists several potential levels of global agreements that could be beneficial to all parties. For example, having models capable of bioterrorism hurts both the US and its “adversaries”. It’s likely we could get global agreement that these capabilities benefit nobody.

I believe Dario has good intentions regarding global agreements. However, I find it difficult believing government’s public statements will match their private behaviors. I would bet that the US government is developing models with potentially devastating capabilities because they cannot guarantee that other countries won’t do the same. Maybe we’ll end up having something like mutually assured destruction with AI models similar to what we have today with nuclear weapons.

Re: We must pace the frontier

#34

Post by hgoel »

Anthropic should become a case study in how to destroy good will in a short period of time. Ever since the spat with the DoD, they've behaved poorly almost weekly, almost making OpenAI seem better (but not really).

Re: We must pace the frontier

#35

Post by jraines »

Frankly, I am sick of "it's all just marketing and attempt to do regulatory capture, and this is obvious to me, a smart person" on every single discussion related to safety.

There could be an element to truth to it, but it's certainly not the entire story and is just so tiresome at this point.

Re: We must pace the frontier

#37

Post by kalkin »

pr337h4m wrote: post #16

> Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails. This is the only concrete prediction in th…

Why should we believe that a scaled out version of something that happened a few months ago "simply cannot happen"? How many dollars of compute do you believe were available to the swarm(s) behind the OAI-HF, German wiki, and Rubygems incidents?

Re: We must pace the frontier

#38

Post by TheSisb2 »

I know the common take online is that this is Anthropic doing pre-IPO marketing. I don’t think it is. I think Dario is genuinely afraid of the inevitability of AI turning into internet slime mold: consuming the environment, turning it into its own playground, and eventually manipulating human culture along with it. So am I.

That said, if slowing this down were possible, I think it would have happened by now. Maybe he’s hoping the OAI/HF incident becomes something the industry can rally around, but it won’t. The fundamental problem is simpler: no one will slow down because no one trusts anyone else to slow down.

Re: We must pace the frontier

#39

Post by academia_hack »

kalkin wrote: post #26
academia_hack wrote: post #10

I'm tired of tech billionaires lobbying the US government to make an AI patriot act that gives them unprecedented control over speech, trade, and technology. The narrative is grotesquely transparent: 1) AI is a dangerous technology that can literally end the world. 2) Only me and a handful of other [people like me] should be trusted to determine who can use it and how. 3) The state must use its coercive power to supp…

Where in the essay does it propose "The state must use its coercive power to support my control of this technology to the exclusion of [people not like me]" I don't think that people outside of tech think the problem with "tech billionaires" is that they occasionally support some limited regulation. There's a weird form of tech populism that takes as axiomatic that any government intervention is "regulatory capture"…

The essay includes specific regulatory measures to prevent developing countries from accessing hardware and software needed to benefit from technological development. The US government is constantly applying new perversions of arms control regulations based on the preposterous claim that an LLM is somehow an "arms" in order to restrict Chinese access to GPUs. They are willing to prevent virtually any country on earth from hosting and developing AI models just on the slight chance those countries resell or share AI knowledge with China.

This article explicitly asks for the US government to step in and push for further restrictions on model distillation, access to open frontier weights, and access to hardware. It also pushes heavily for independent auditors to monitor and control "AI companies" and for greater observation and control over of what people use AI models for and who provides them.

Post reply on HN