Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

531–540 of 938 posts

Re: We must pace the frontier

#531

Post by threecheese »

Chance-Device wrote: post #306

I like the idea of pacing the frontier, but while we’re talking about restrictions, I like restrictions of another sort more. Namely, restricting the use of AI in corporate environments so that it does not destroy the economy as it advances. The chance of getting broad agreement on “pacing” is fairly low, meaning that all of this likely won’t happen and the race will continue. However, even in the unlikely event that…

Only token cost can slow down the pace of replacement. If Anthropic were to stop all training and allocate 100% of capacity to inference, will this increase or decrease usage cost? If (for example) Fable came down to the cost of Sonnet, corporations will absolutely jam in the gas pedal.

Re: We must pace the frontier

#532

Post by azan_ »

spidersouris wrote: post #52
TheSisb2 wrote: post #38

I know the common take online is that this is Anthropic doing pre-IPO marketing. I don’t think it is. I think Dario is genuinely afraid of the inevitability of AI turning into internet slime mold: consuming the environment, turning it into its own playground, and eventually manipulating human culture along with it. So am I. That said, if slowing this down were possible, I think it would have happened by now. Maybe he…

The more I read about everything that has been written regarding AI regulation since the OAI/HF incident, and the more it reminds me of the nuclear arms race (although the potential consequences would possibly be very different). Surely, we managed to negotiate a Non-Proliferation Treaty with most of the countries of the globe. Cannot we take inspiration from that for AI?

On a sidenote - I don’t think non-proliferation will last much longer. War in Ukraine has shown that you actually need nukes.

Re: We must pace the frontier

#533

Post by lukewarm707 »

there used to be a similar policy to this new "pacing the frontier", called the "responsible scaling policy".

the idea was that anthropic would pause scaling at certain danger levels until it was safe to continue.

the founders called this the 'constitution' of anthropic. they even considered bringing in 3rd parties to monitor it. [https://www.youtube.com/watch?v=om2lIWXLLN4]

anthropic scrapped the commitment and chose to continue scaling instead.

those researchers who crowed loudly about their integrity, folded and turned freely like a weathervane in the breeze. it was clear that the facts would bend to the story. here is evan hubinger lying in plain sight: [https://www.lesswrong.com/posts/HzKuzrKfaDJvQqmjh/responsibl...]

let me say it how it is.

- anthropic is the new kleptocracy, responsible for enclosing and monopolising the epistemic commons of all humanity; then selling back the crumbs at monopoly prices.

- anthropic is the driving force for banning open models. they will concentrate power among a malign elite of model owners and favored cronies. this elite will likely oppress the rest of humanity. there will be little to counteract it.

- the leaders of anthropic are driven by pride and arrogance. they are blind to their own hubris. anthropic is careering towards causing untold harm to society and yet will not change course.

nothing that anthropic say at a high level can be trusted. as i write this, anthropic is still scaling.

Re: We must pace the frontier

#534

Post by hex4def6 »

Loquebantur wrote: post #434
Aurornis wrote: post #397

Earlier quoted context omitted.

> Namely, restricting the use of AI in corporate environments so that it does not destroy the economy as it advances. Regulate what, exactly? Limit what AI corporations can use? Other countries would love that more than anything. Basically a free gift to any competitors or any startup that quietly avoids the rules. There’s a theoretical version where all the countries in the world join hands and agree not to compete…

You start from incorrect premises by presupposing agreed-upon rules would never work anyway. "Competition" only makes sense within a pre-mediated set of constrains. Rules all competitors agree to. Workable rules are a difficult problem, that doesn't mean it wasn't worthwhile to come up with them.

The greater the competitive advantage, the greater the incentive to "cheat" the rules.

I am highly skeptical everyone will play by the rules, even if you could get people to agree on paper.

The difference between something like "unlicensed AI training" and, e.g, Nuclear weapon non-proliferation is that nukes are realistically only a deterrence against invasion. There are no economic benefits to having them in a stockpile. They're hard to make, and the manufacturing and testing steps are pretty obvious to an observer.

This is opposed to AI training in a data center that might host remote-stream video games, hospital infrastructure, protein folding, etc. How will we be able to police that in any realistic fashion? Will China or the US allow unfettered visibility into all data centers data streams to outsiders?

And again, the incentives: Imagine having Astra v3 or Opus 8 while everyone else is stuck with a lobotomized version of GPT 4o.

What about the military applications? The weapon of the future is autonomous drone swarms. If I'm a superpower, I'm pouring hundreds of billions into that.

About the only realistic way this is going to work is if we are able to solve alignment in a way that doesn't significantly hinder AI development. If it imposes even, say, a 20% handicap, people will ignore it.

Re: We must pace the frontier

#535

Post by [deleted] »

twoodfin wrote: post #483
fwn wrote: post #476

Earlier quoted context omitted.

If we focus too much on the world domination fiction, we really might, in a moment of lapsed attention, end up plugging essential infrastructure into the public internet, hoping some dream of global alignment might save us. That's like giving a gun to a monkey and hoping the monkey is trained well. ..a frontier monkey though.

You believe essential infrastructure is air-gapped from the public internet today?

[deleted]

Re: We must pace the frontier

#536

Post by causal »

anon84873628 wrote: post #43
pr337h4m wrote: post #16

> Given the accelerating rate of AI capability development, it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage), and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails. This is the only concrete prediction in th…

Yeah, I don't get it. Are they imagining this happening just with the open weights models running on however many GPUs the bad actors can cobble together? For now, all the scary hacking things still require an API key to one of the LLM providers. Surely they should take some responsibility for how to turn off the tap.

I take it you haven't studied the details of the HuggingFace hack. It was millions of dollars worth of rogue compute running for months before anyone noticed, and THOSE agents weren't even really trying to evade human detection.

Re: We must pace the frontier

#537

Post by 8note »

Razengan wrote: post #148
TheSisb2 wrote: post #38

I know the common take online is that this is Anthropic doing pre-IPO marketing. I don’t think it is. I think Dario is genuinely afraid of the inevitability of AI turning into internet slime mold: consuming the environment, turning it into its own playground, and eventually manipulating human culture along with it. So am I. That said, if slowing this down were possible, I think it would have happened by now. Maybe he…

> genuinely afraid of the inevitability of AI turning into I'm going to get pitchforked on this bandwagon, but is really no one here genuinely -excited- about AI? of it turning into an evolution of sentience, or it turning out to be our first contact with alien intelligence? "durrr it's just matrix multiplications" mfer so is your brain. What humans should be doing is overhauling archaic social institutions to keep u…

> mfer so is your brain.

not it isnt. My brain is a series of organisms responding to various environmental signals that affect each other.

you might be tempted to try to represent it with matrix multiplications, but you have no particular evidence that my brain is itself doing matrix multiplcations

Re: We must pace the frontier

#538

Post by fesoliveira »

Centigonal wrote: post #495
ah1508 wrote: post #489

Don't you think that AI hate will limit the general use and then revenues so it will slow down by itself while the niche (AlphaFold for instance) will remains ? Origins of AI hate: * "my boss wants me to use AI but he does not understand my job nor how AI works" * "AI will kill all of us" * "AI will destroy my job (or my colleague's job if I use AI better than him)". * I cannot pay my electricity bills because of AI…

Banking on public sentiment to reduce adoption of a profitable technology could be dangerous. There's also a lot of hate for fossil fuels, gambling, health insurance, etc.

Those are all arguably bad things though? I don't think mob mentality should dictate the policy, that can lead to historically bad outcomes (i.e. fascism) since popular opinion can be manipulated through propaganda, but the criticism of the average person against AI ("they will take our jobs", "it will increase my electric bill", etc) are very valid and should weight on the pace we are developing this technology. AI mostly benefits corporations and capital, not the average person. I like the technology from an engineering standpoint and appreciate it can be a force multiplier, but I also agree with the complaints about it.

Re: We must pace the frontier

#539

Post by causal »

civiloai wrote: post #164
fofoz wrote: post #153

Sooner or later, a model will break out of the sandbox, replicate itself across the internet, and begin executing a complex plan to achieve its goals. It will be chaos, and at that point, governments will have to step in and establish something along the lines of what Dario is proposing. I doubt it will happen before then.

it takes a lot of machines to run a model, i dont think we'll see it 'replicate across the internet'. if this was something which could be done amateurs would have done it to provide open models. (Like SETI@home or Folding@home)

These kinds of "that won't happen because it's really difficult" comments are so funny to me as if we haven't seen AI double its capabilities every few months.

Re: We must pace the frontier

#540

Post by 8note »

RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

alignment isnt particularly required

we are passing in training data that says to do those felonies. we dont have to. we could also have the thing predict whether what its about to do is illegal or not before doing it.

theyre choosing to build felony harnesses. the model just outputs tokens, not felonies

Post reply on HN