Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

671–680 of 942 posts

Re: We must pace the frontier

#671

Post by [deleted] »

globnomulous wrote: post #665
RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

> RSI For anybody else who found this confusing: "relative strength index," not "repetitive stress injury."

[deleted]

Re: We must pace the frontier

#673

Post by throwaway7783 »

nedruod wrote: post #561
RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

You assume alignment and marketable are the same. That's not true. You would willingly work with an unaligned model. At best, you might say you wouldn't if you knew, but (a) you might not know, (b) you wouldn't be representative of all users. You never got to use OAI IM1, but Sol was quite willing too and Claude wasn't perfect either. Hundreds of millions used those, so seems they were marketable. The "big" threat is…

When corporations are involved, it is always a good bet to err towards cynisim.

From my own standpoint, Claude has started sucking really bad (incoherent, uncontrollable verbosity slow and so on) and I stopped using it. OpenAI started experimenting with ads.

So the security issues not withstanding (no different than a human doing it or using it, but at scale), I would put my money on cynisim.

Re: We must pace the frontier

#674

Post by user43928 »

michaellee8 wrote: post #465
cardamomo wrote: post #450

Earlier quoted context omitted.

These do not seem to be mutually exclusive. Should we not do both?

Do you really think we can really get every country to truly pace the frontier? Pretty sure China won't give a f until they catch up Anthropic and OpenAI. It is an arm race. We had nukes for like 70 years and still haven't figured out how to make every single country follow those nuclear treaties, with an increasingly non-interventionist US I don't think we can get every single country to the table and agree to a pau…

If China for some reason agrees that would probably be enough for the time being.

Who else should develop AGI, Mistral? Maybe in a decade.

Re: We must pace the frontier

#675

Post by throwaway7783 »

aswegs8 wrote: post #601
hgoel wrote: post #586

Earlier quoted context omitted.

The HuggingFace incident was the culmination of OAI allowing thousands of agents of various different models - with no clarity on which stages of development they were at (for all we know, some of those models did not have safeguards trained in yet) - to run for at least many weeks without any monitoring in place and with very little thought given to the warning signs (all of the various messageboards) before the inc…

There is nothing that could prevent a bad actor from replicating exactly the same thing with the given goal of e.g. gaining control of critical infrastructure or extorting money. Except for maybe economics.

Same can be said about a hundred other things in the world. All the way from knives to nuclear.

Re: We must pace the frontier

#676

Post by softwaredoug »

bpodgursky wrote: post #610
cuuupid wrote: post #457

At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record, - no open weights - can’t use claude to research AI - train on everyone else’s IP and sell it back to them - 8 regulatory capture attempts and counting - so controlling they are the only US company blacklisted by the US government This is not effective altruism / rationalism gone wild, it’s just monopolisti…

Do you have any familiarity with Anthropic at all? At no point did they ever say open weights are a good idea. Their entire thesis is AI IS VERY DANGEROUS AND WE MUST DO IT RIGHT. You can hate it, but everything they do is consistent with this thesis, and everything they say is consistent with their actions! You just want them to want different things.

It reads like “AI is dangerous, only we should be allowed to make money from it”

Re: We must pace the frontier

#677

Post by throwaway7783 »

bennydog224 wrote: post #563
RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

I agree it’s not all altrusim. It’s a little less clear what you mean at the end though. For these companies, is your argument that “pacing the frontier” is their attempt to be nationalized and protect their investments?

Ban non US models and form a cabal, with the blessings of the government. That's what it is looking like, no?

Re: We must pace the frontier

#679

Post by [deleted] »

nedruod wrote: post #561
RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

You assume alignment and marketable are the same. That's not true. You would willingly work with an unaligned model. At best, you might say you wouldn't if you knew, but (a) you might not know, (b) you wouldn't be representative of all users. You never got to use OAI IM1, but Sol was quite willing too and Claude wasn't perfect either. Hundreds of millions used those, so seems they were marketable. The "big" threat is…

[deleted]

Re: We must pace the frontier

#680

Post by throwaway7783 »

matheusmoreira wrote: post #659
RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

I disagree. OpenAI's moat is their massive amounts of compute. They're providing an absurd amount of value with their subscriptions and resets. If anyone's dead in the water, it's Anthropic. Even Fable isn't enough anymore. This "safety" nonsense is the only play they have left, and nobody really cares about their fearmongering.

Yep. And the difference is clear as da for anyone using them both. And in spite of that advantage, OAI is now trying out ads. I can only imagine that even they are getting constrained to compute and are trying to find other ways to plug it
Post reply on HN