Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

841–850 of 946 posts

Re: We must pace the frontier

#841

Post by SturgeonsLaw »

alexfortin wrote: post #830
pigeons wrote: post #812

Earlier quoted context omitted.

Because you basically get a discount to use claude code via subscription when using an anthropic model, compared to what you pay via api billing with another harness

Understood. Personally that's actually another good reason to boycott Anthropic: beside the fact I perceive their models as (at best) marginally better than the ones I'm used to (Z.ai glm-5.3-flash, DeepSeek Flash v4.1), they even force me to use their bloated harness. They are not even open weights and iirc they're even encrypting chain of thoughts now? Litterally, from my perspective there seems to be no reason wha…

I too am using GLM-5.3-flash in Pi and I've yet to encounter a scenario it couldn't handle. And the pricing is just incredible, I've handed it a previously unseen codebase, asked it to analyse it and build a new feature, came back after it had done so and the API cost was a fraction of a cent. It's $0.5/1M output tokens on OpenRouter.

If I really need to, I can escalate a task to Opus at $25/1M, and the results are good, but not 5000% as good.

Re: We must pace the frontier

#842

Post by keeda »

hgoel wrote: post #554
RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

Why are we accepting the framing that the LLMs are felony generators, when the only incidences of LLM generated felonies involved misconfigured sandboxes and reckless waste of resources? The companies doing these things without following common sense security measures are the felony generators.

As TFA calls out, these agents were not asked to do any of these things and yet they did, at a bonkers scale, within just this handful of companies you mention. Whether they had leeway to is secondary to the fact that they did.

Heck, they exploited zero day flaws which by definition means they went beyond common sense security measures.

And now these agents are already being deployed all over the world at an ever increasing pace. How much of the world do you think follows "common sense security measures"?

Re: We must pace the frontier

#843

Post by ozozozd »

braydenm wrote: post #270

Before Anthropic, I worked at Cruise for four years as it competed against Waymo. The culture rewarded (and demanded) moving quickly, trusting that the company could empirically discover the risks that the robot cars posed and iteratively solve them to keep up with the rate at which it was scaling out its technology. There was very little interest or appetite for coordinating or collaborating with other AV companies…

Have you quit? Are you under duress? If you haven’t quit, or under duress, your comment reads very strange.

If I had even a tiny doubt that I didn’t have many “Saturdays I’ll be able to play outside with my kids,” I wouldn’t show up to work anymore.

If you have these concerns, and still planning to show up to work on Monday, you are either insincere, or have terrible judgement.

Re: We must pace the frontier

#844

Post by jimbokun »

FusionX wrote: post #782
RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

We're already seeing anti-AI sentiments, but the movement is still fringe with a vocal minority. However, that'll change soon without alignment. Without self-intervention, there will invariably be future incidents that can cause major economic impact, leaked private data, loss of life (directly/indirectly) etc. Once that happens, their social capital is wiped. It'll be an avalanche of lawsuits and overzealous regulat…

Anything related to AI is extremely unpopular right now with the general public.

Re: We must pace the frontier

#846

Post by adsharma »

anon291 wrote: post #839
adsharma wrote: post #501

So many words. But missing the one that matters the most: Explainability. AI slowdown is worth it only if it can be made more explainable. Changing the language we use to discuss it is a good first step. We need to stop using inside baseball terms like alignment and mechanistic interpretability. Replace them with explainable tech. Graph Databases, Causality, Shared semantic spaces. Previous writings on the topic (als…

As much as I love all these technologies, they don't have any direction that approaches the efficacy of a transformer

You can have a transformer based index built on top of a graph database. Much of the explainability tech (people who prefer "alignment" also prefer "mechanistic interpretability") is reverse engineering the real world graph that's hiding in the weights.

https://research.google/pubs/the-case-for-learned-index-stru...

Re: We must pace the frontier

#847

Post by ozozozd »

bwfan123 wrote: post #278
blfr wrote: post #8

I enjoy Claude Code very much, have max privately and team premium at work, but the doomer marketing and this whole regulate-while-we're-ahead spiel is extremely annoying and makes me wish Anthropic gets trounced.

> the doomer marketing It is not doomer marketing - it is clever psy-ops magic trick. 1) Software is described in super-human terms when it is plain-old software. Was stockfish described like this ? no. 2) Datacenters become AI-factories. 3) Hardware becomes investible assets. 4) Malware becomes a super-human breakout (oai/hf) All clever framing to market the new technology. If it is called for what it really is ie,…

Agreed!

George Carlin has an amazing show about how language has changed over his lifetime in a different way.

You would certainly enjoy it.

Re: We must pace the frontier

#848

Post by missedthecue »

madrox wrote: post #588

This may come out of left field, but Dario seems terrified to be in charge. I don't get the impression that he ever had a desire to run a company like this. Now that he's a CEO, he keeps trying to make uncompetitive decisions and calls for someone (anyone) to stop him. It regularly blunts Anthropic's edge. It's the only explanation I can see when it's obvious to any student of history this is going to backfire. It do…

I have been saying over and over again that the board ought to fire him (or move him to a nonconsequential "advisory role"). I totally agree that he seems mostly directionless and far more interested in the academic side of things than the product side of things.

Re: We must pace the frontier

#849

Post by Miraste »

deepwoods wrote: post #741
barrrrald wrote: post #733

Earlier quoted context omitted.

I have many issues with Anthropic, but I will say that their actions are fully consistent with a group of people who earnestly believe that AI is extremely dangerous In fact, I would say that the Occam’s razor explanation is not that they are seeking regulatory capture, but that they earnestly believe in the x-risk, and that they are the most thoughtful and capable people to address it You may disagree, you may think…

I agree with this take. Regardless of whether you think they're right, most everyone at Anthropic is acting in good faith. If this is an intentional media campaign it is a remarkably sloppy one. It has been effective because if you tell people they're going to die, they tend to pay attention - think of the grip the 2011 Harold Camping rapture prediction had on our collective psyche, or the 2012 apocalypse. But the me…

I think you're right, and that's worse.

This disaster of a PR strategy is not going to produce the outcomes Dario says he wants. It's going to make people hate him. Most importantly for his goals, it's going to make Chinese labs hate him--and they already hate him, because he treats them as basically terrorists. So if there is a "pacing", it won't include China, and thus may as well not happen.

That leaves only two conclusions, and really only one:

Dario believes what he says about safety, but does not understand politics and is prone to very counterproductive action, and therefore can't be trusted at the helm of a leading AI company.

or

Dario is lying about his goals, and therefore also can't be trusted.

Re: We must pace the frontier

#850

Post by camkego »

If Anthropic really wants to make a statement they could independently pace their own model development, and ask others to make the same pledge.

Somehow, I suspect that won't happen.

Post reply on HN