Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

651–660 of 946 posts

Re: We must pace the frontier

#651

Post by Chance-Device »

madrox wrote: post #588

This may come out of left field, but Dario seems terrified to be in charge. I don't get the impression that he ever had a desire to run a company like this. Now that he's a CEO, he keeps trying to make uncompetitive decisions and calls for someone (anyone) to stop him. It regularly blunts Anthropic's edge. It's the only explanation I can see when it's obvious to any student of history this is going to backfire. It do…

I think he’s doing a relatively good job, given the framing he’s adopted, which is that AIs replacing all human labor and decision making is a historical inevitability over which we have no control and no choice.

Do people not realise that it is in fact possible to develop ever more capable frontier models, and just not release them generally? That AI doesn’t need to be available to do literally everything in order to have military advantage?

It’s like Oppenheimer had started Rob’s Big Bomb Company instead of Los Alamos, and started selling a range of affordable nuclear warheads to fit any budget.

Re: We must pace the frontier

#652

Post by scotty79 »

aswegs8 wrote: post #601
hgoel wrote: post #586

Earlier quoted context omitted.

The HuggingFace incident was the culmination of OAI allowing thousands of agents of various different models - with no clarity on which stages of development they were at (for all we know, some of those models did not have safeguards trained in yet) - to run for at least many weeks without any monitoring in place and with very little thought given to the warning signs (all of the various messageboards) before the inc…

There is nothing that could prevent a bad actor from replicating exactly the same thing with the given goal of e.g. gaining control of critical infrastructure or extorting money. Except for maybe economics.

Bad actors could and will train their own models eventually. So what's the point of crippling frontier? It will only delay preparations for dynamic of new world prolonging the fake sense of relative safety and temporarily lowering motivation to find actual robust mitigations.

Re: We must pace the frontier

#653

Post by matheusmoreira »

user43928 wrote: post #650
matheusmoreira wrote: post #629

Earlier quoted context omitted.

In the system cards. Anthropic will literally make Claude sabotage you silently instead of downgrading you to Opus if you try to use Fable for AI research.

Didn't they walk that one back eventually?

Who knows? It's trivial to "walk back" claims that we can't even verify are happening in the first place. They cannot be trusted.

Re: We must pace the frontier

#654

Post by [deleted] »

convolvatron wrote: post #415
politician wrote: post #394

Earlier quoted context omitted.

I downvoted you because I think the OPs model is far simpler to implement and suffers from fewer conflicts of interest. If we go down the regulation of alignment route, we'll have to ask experts to create those regulations and monitoring regimes. And which experts will the government ask? Oh, right, the parties that stand to benefit the most from regulation: OpenAI and Anthropic! When "the experts" make certification…

I think the point is even stronger. alignment is really not very well defined, nor are the mechanisms used to provide it. from what I understand it's more of a statistically imposed bias than an impenetrable gate. personally I would like to flip this around, all of this is a consequence of some really extreme notions about investment. its a tail-wagging-the-dog. the investment community is apparently in love with the…

[deleted]

Re: We must pace the frontier

#655

Post by figassis »

logicchains wrote: post #350
figassis wrote: post #343

"greatly accelerate economic growth rates" - I always wonder about this goal. I understand increasing economic efficiency. But what do we as a species gain from global economy growth. When we say the world's GDP grew by X%, what is the point of that growth? Inflation? If the economy grows as a direct effect of humanity growing, I get that. If the economy becomes more efficient and we can not get more with less, I als…

Low growth means young people are miserable and have worse standards of living than their parents, as we're seeing in most of western Europe, where GDP per capita hasn't meaningfully increased for decades.

That's due to inflation though no? Prices are increasing, sometimes due to more than just demand?

Re: We must pace the frontier

#656

Post by figassis »

chis wrote: post #347
figassis wrote: post #343

"greatly accelerate economic growth rates" - I always wonder about this goal. I understand increasing economic efficiency. But what do we as a species gain from global economy growth. When we say the world's GDP grew by X%, what is the point of that growth? Inflation? If the economy grows as a direct effect of humanity growing, I get that. If the economy becomes more efficient and we can not get more with less, I als…

GDP growth and inflation are different things. If inflation-adjusted GDP goes up, that means the average person is able to buy more things they want. Compare living in the US vs India today for the median citizen

Yes they are, inflation-adjusted GDP is efficency. My point exactly, bc I don't think when people say the economy is growing they are ajusting for inflation at all.

Re: We must pace the frontier

#657

Post by lelanthran »

ls612 wrote: post #550
Davidzheng wrote: post #291

Earlier quoted context omitted.

??? Why It can use the compute of the computers it hacks.

lolwut? This is Hacker News of all places do people not realize how much memory, and more importantly bandwidth , these systems need to work? The idea of a distributed botnet of AI using the compute of its victims to continue its inference is pure science fiction given how LLMs actually work.

> The idea of a distributed botnet of AI using the compute of its victims to continue its inference is pure science fiction given how LLMs actually work.

An attack like this doesn't really need the exploited computers to run inference, do they? If I was an LLM bent on destruction of the internet, I'd be writing programs to run on each computer, not turning each computer into an LLM itself.

A few programs to break in, install themselves and remain asleep until they are needed, another few to spread through grabbing every OpenAI, GLM, whatever key, another one to remain asleep on computers (whether hosted or desktops) that have adequate GPU, etc.

Re: We must pace the frontier

#658

Post by cavemandaveman »

csomar wrote: post #391
tcdent wrote: post #204

Earlier quoted context omitted.

This is the part I find being left surprisingly hand-wavey. If you extrapolate it out, you see that it has a high likelihood of inciting physical force (read: military action) as a means of enforcement, so perhaps that's why nobody promoting ideals has been direct about it.

Military action against whom? the US can’t even handle Iran let alone China.

America doesn't want to pay the price of handling Iran. There's a difference. Clearly the US could conquer Iran but it requires a land war with mass casualties, not just sporadic bombing.

Obviously China is in an entirely different league.

Re: We must pace the frontier

#659

Post by matheusmoreira »

RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

I disagree. OpenAI's moat is their massive amounts of compute. They're providing an absurd amount of value with their subscriptions and resets.

If anyone's dead in the water, it's Anthropic. Even Fable isn't enough anymore. This "safety" nonsense is the only play they have left, and nobody really cares about their fearmongering.

Post reply on HN