Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

811–820 of 943 posts

Re: We must pace the frontier

#811

Post by tonetheman »

FusionX wrote: post #782
RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

We're already seeing anti-AI sentiments, but the movement is still fringe with a vocal minority. However, that'll change soon without alignment. Without self-intervention, there will invariably be future incidents that can cause major economic impact, leaked private data, loss of life (directly/indirectly) etc. Once that happens, their social capital is wiped. It'll be an avalanche of lawsuits and overzealous regulat…

Not true. Anti AI is not a minority or fringe.

Outside of my tech people I know no one who thinks highly of AI.

Re: We must pace the frontier

#812

Post by pigeons »

alexfortin wrote: post #778
throwaway7783 wrote: post #673

Earlier quoted context omitted.

When corporations are involved, it is always a good bet to err towards cynisim. From my own standpoint, Claude has started sucking really bad (incoherent, uncontrollable verbosity slow and so on) and I stopped using it. OpenAI started experimenting with ads. So the security issues not withstanding (no different than a human doing it or using it, but at scale), I would put my money on cynisim.

I'm curious, what are the reasons to use Claude Code anymore when there are so many other (allegedly better) OpenSource harnesses out there? Personally I've been using https://pi.dev for long and never looked back.

Because you basically get a discount to use claude code via subscription when using an anthropic model, compared to what you pay via api billing with another harness

Re: We must pace the frontier

#814

Post by skue »

Chance-Device wrote: post #306

I like the idea of pacing the frontier, but while we’re talking about restrictions, I like restrictions of another sort more. Namely, restricting the use of AI in corporate environments so that it does not destroy the economy as it advances. The chance of getting broad agreement on “pacing” is fairly low, meaning that all of this likely won’t happen and the race will continue. However, even in the unlikely event that…

There are serious people trying to figure out what that looks like. Check out the recent Freakonomics episode with Gina Raimondo.

Re: We must pace the frontier

#815

Post by RGS1811 »

adsharma wrote: post #785
RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

The real threat is that we uncritically adopt language such as alignment. Implicit in this is the idea that AI is a inscrutable matrix and going to remain that way and we'll need expert interpreters to make sense of it. We need to insist on building tech that's explainable by design.

Alignment just means "this machine operates in ways that align with the intent of its users". It doesn't imply anything about the inscrutability of the machine in question. A gun with a misaligned scope would likewise fail to operate in accord with its user's intent, and likewise with potentially deadly consequences.

Re: We must pace the frontier

#816

Post by throwaway7783 »

afthonos wrote: post #797
throwaway7783 wrote: post #706

Earlier quoted context omitted.

The cynicism is about motivations and not that they are inherently not bad. Perhaps they are as bad as they claim. Or perhaps they're worse. All we have a couple of run of the mill breach examples and some people inside the talking about how dangerous it is. Yes, they are far more qualified than I am (or most people here), and perhaps there is a grain of truth. It is the motivation - and it is always money with corpo…

It is always money — but it isn’t always only money. They are not asking for anything that will prevent them from making money in the future, but they are asking for help stopping the runaway train they’re on. These are compatible requests.

All the whistleblowers have said the alignment projects are underfunded. Surely they don't need external help to fund that? Could've done years ago?

Re: We must pace the frontier

#817

Post by adsharma »

RGS1811 wrote: post #815
adsharma wrote: post #785

Earlier quoted context omitted.

The real threat is that we uncritically adopt language such as alignment. Implicit in this is the idea that AI is a inscrutable matrix and going to remain that way and we'll need expert interpreters to make sense of it. We need to insist on building tech that's explainable by design.

Alignment just means "this machine operates in ways that align with the intent of its users". It doesn't imply anything about the inscrutability of the machine in question. A gun with a misaligned scope would likewise fail to operate in accord with its user's intent, and likewise with potentially deadly consequences.

The gun comes with a manual on how to use it safely. I'm sure it has some complexities, but at the high level:

> A gun is a metal tube that uses a tiny, controlled explosion to shoot a small piece of metal (called a bullet) forward at very high speed.

If the gun doesn't work as intended, you can take it to a shop and someone can fix it so it works as designed.

All I'm saying is AI should be designed the same way. Treat AI as normal tech like any other and use similar language.

Re: We must pace the frontier

#818

Post by deepwoods »

ElProlactin wrote: post #787
deepwoods wrote: post #741

Earlier quoted context omitted.

I agree with this take. Regardless of whether you think they're right, most everyone at Anthropic is acting in good faith. If this is an intentional media campaign it is a remarkably sloppy one. It has been effective because if you tell people they're going to die, they tend to pay attention - think of the grip the 2011 Harold Camping rapture prediction had on our collective psyche, or the 2012 apocalypse. But the me…

> Regardless of whether you think they're right, most everyone at Anthropic is acting in good faith. The road to hell is paved with good intentions. Dario Amodei is a 40-something dude who is obviously very, very intelligent. But intelligence is not wisdom and his "essay" here can easily be read as someone who opened a can of worms and doesn't know (or can't accept) that he won't be able to put the worms back in. But…

I see what you mean. I meant more that they genuinely believe the tech is dangerous, not that they are necessarily doing the right thing.

Re: We must pace the frontier

#819

Post by chillfox »

barrrrald wrote: post #733
cuuupid wrote: post #457

At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record, - no open weights - can’t use claude to research AI - train on everyone else’s IP and sell it back to them - 8 regulatory capture attempts and counting - so controlling they are the only US company blacklisted by the US government This is not effective altruism / rationalism gone wild, it’s just monopolisti…

I have many issues with Anthropic, but I will say that their actions are fully consistent with a group of people who earnestly believe that AI is extremely dangerous In fact, I would say that the Occam’s razor explanation is not that they are seeking regulatory capture, but that they earnestly believe in the x-risk, and that they are the most thoughtful and capable people to address it You may disagree, you may think…

I don't think so.

If they actually believed in it, then they would stop pushing the frontier of capability and instead focus on alignment, safety, and better tools for controlling/debugging AI. Then they would actually share their findings and tools.

They don't do this. Instead of reducing the competitive pressure and helping the industry to build safer more aligned models they are doing the exact opposite.

Re: We must pace the frontier

#820

Post by anon291 »

augment_me wrote: post #638
Chance-Device wrote: post #306

I like the idea of pacing the frontier, but while we’re talking about restrictions, I like restrictions of another sort more. Namely, restricting the use of AI in corporate environments so that it does not destroy the economy as it advances. The chance of getting broad agreement on “pacing” is fairly low, meaning that all of this likely won’t happen and the race will continue. However, even in the unlikely event that…

This is a call to end capitalism. Its completely impossible to do what you suggest with the incentives of capitalism.

Forget capitalism. This is a call to end the basic right of people to multiply matrices
Post reply on HN