Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

861–870 of 942 posts

Re: We must pace the frontier

#861

Post by alach11 »

chillfox wrote: post #819
barrrrald wrote: post #733

Earlier quoted context omitted.

I have many issues with Anthropic, but I will say that their actions are fully consistent with a group of people who earnestly believe that AI is extremely dangerous In fact, I would say that the Occam’s razor explanation is not that they are seeking regulatory capture, but that they earnestly believe in the x-risk, and that they are the most thoughtful and capable people to address it You may disagree, you may think…

I don't think so. If they actually believed in it, then they would stop pushing the frontier of capability and instead focus on alignment, safety, and better tools for controlling/debugging AI. Then they would actually share their findings and tools. They don't do this. Instead of reducing the competitive pressure and helping the industry to build safer more aligned models they are doing the exact opposite.

> If they actually believed in it, then they would stop pushing the frontier of capability and instead focus on alignment, safety, and better tools for controlling/debugging AI. Then they would actually share their findings and tools.

I don't think this is fair. They've published more about alignment and safety than any other AI lab. And in the absence of a coordinated pause, them pausing development unilaterally doesn't do much to advance AI safety.

Re: We must pace the frontier

#862

Post by tclancy »

tclancy wrote: post #860
tclancy wrote: post #857

Earlier quoted context omitted.

I think it gets easier if you stop conflating getting investment with having a goddamn clue or a moral backbone. Occam’s Razor for this dude, Sam Altman, or anyone else: if I said, “some moron on a a street corner just said …” would that change your take on the words? Because I think a lot of what we are hearing is a bunch of people who never ever had to deal with a single consequence all of a sudden worry there migh…

“ I have worked on AI for the last twelve years because I believe it could dramatically raise the quality of human life” there you go. What if an utter idiot had done and said that? First, is it impossible to believe an idiot who didn’t need to work to live might do such a thing? If not, is it impossible to believe they would wind up here, barfing their externalities onto us?

“ believe that AI could cure most major diseases in the next 5–10 years,”

I do not have the least bit of idea how disease works but I am sure the hammer I am working on will nail it all.

If your famously atemporal agents can solve disease, why would it happen over a timeline? Wouldn’t they just figure it out and then … well at that point either tell us or, given the attacks on ruby gems, et al we have seen from agents with “misconfigured” goals, they’d still tell us how to cure the pox they invented, right?

Re: We must pace the frontier

#863

Post by snaking0776 »

wslh wrote: post #708
snaking0776 wrote: post #527

Earlier quoted context omitted.

Yes I think this level of pessimism doesn’t actually hold up to the history of the last 100 years as it relates to nuclear weapons, cloning, wars etc. In fact I think the history actually points to the fact that believing such deals were impossible was a source of much of the strife. In many cases as soon as people started to actually speak to each other about what they had been fighting about they quickly realized t…

> Agreeing to end nuclear testing saved us from the Cuban Missile Crisis. The treaty came after the crisis, not before. And it only banned atmospheric, underwater, and space tests. The US and USSR kept testing underground on a large scale, while France and China never signed it and continued atmospheric tests for years.

Fair, I was going off memory and was wrong. I guess I was mostly just trying to get across that communication seems to be more powerful than we give it credit for, but should’ve double checked.

Re: We must pace the frontier

#864

Post by tclancy »

adsharma wrote: post #785
RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

The real threat is that we uncritically adopt language such as alignment. Implicit in this is the idea that AI is a inscrutable matrix and going to remain that way and we'll need expert interpreters to make sense of it. We need to insist on building tech that's explainable by design.

That last line needs a lot of workshopping. A guillotine with instructions on the bottom of the blade conforms to your request.

Re: We must pace the frontier

#865

Post by altpaddle »

academia_hack wrote: post #90

Dario's proposed approach is a classic example of capital attempting to control technological advancement and the means of production. For the first time in human history, any member of the working class can just about afford to have a team of expert scientist/physician/lawyer/engineers working directly for them. Super intelligence (the ability to have many smarter minds than your own reporting to you) has always bee…

What a foolish comment, LLMs empower capital owners far more than the 'working class'

Re: We must pace the frontier

#866

Post by RobertDeNiro »

aubanel wrote: post #777
cuuupid wrote: post #457

At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record, - no open weights - can’t use claude to research AI - train on everyone else’s IP and sell it back to them - 8 regulatory capture attempts and counting - so controlling they are the only US company blacklisted by the US government This is not effective altruism / rationalism gone wild, it’s just monopolisti…

You can disagree with Anthropic leadership, but all the points you mentioned can reconcile very well with them thinking in good faith "advanced AI is too dangerous to be left in all hands" Except the "train on everyone else IP" which can be said of all AI companies.

Problem with Anthropic is that they benefited massively from open source and now are completely against it. It’s hypocrisy at its worst. Why do they get to benefit, but not everyone else?

Re: We must pace the frontier

#868

Post by zombiwoof »

RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

[dead]

Re: We must pace the frontier

#869

Post by panarky »

FusionX wrote: post #782
RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

We're already seeing anti-AI sentiments, but the movement is still fringe with a vocal minority. However, that'll change soon without alignment. Without self-intervention, there will invariably be future incidents that can cause major economic impact, leaked private data, loss of life (directly/indirectly) etc. Once that happens, their social capital is wiped. It'll be an avalanche of lawsuits and overzealous regulat…

>> I don’t understand all the comments assuming that RSI is the real threat here

> leaked private data, loss of life

This smells like more of a money move than a safety move.

Amodei is proposing to form a cartel of American frontier labs.

They all agree to shift compute away from cash-burning research and training toward cash-generating inference.

Then tacitly agree not to compete on price.

They'll install independent auditors inside each company to ensure nobody cheats.

And back it up with government regulation or diktat to punish defectors from the cartel.

Then they'll lock out non-American labs with export controls and regulations on open-weights models to funnel global inference tokens through their cartel.

It wouldn't be the first time a tech oligopoly used "safety" as the pretext to establish a government-sanctioned cartel.

Railroads and airlines ran this same playbook.

Re: We must pace the frontier

#870

Post by spopejoy »

cavemandaveman wrote: post #658
csomar wrote: post #391

Earlier quoted context omitted.

Military action against whom? the US can’t even handle Iran let alone China.

America doesn't want to pay the price of handling Iran. There's a difference. Clearly the US could conquer Iran but it requires a land war with mass casualties, not just sporadic bombing. Obviously China is in an entirely different league.

> Clearly the US could conquer Iran

Vietnam (and Ukraine) would like a word

Post reply on HN