Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

831–840 of 948 posts

Re: We must pace the frontier

#831

Post by sobrey »

RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

I think the real reason he is asking for pacing, is that in a world were AI becomes rampant, he will be seen as Hitler. I would bet this is mostly self-motivated.

Re: We must pace the frontier

#832

Post by tom2026hn »

If you think everyone is copying you, then by not releasing more advanced models, you can significantly slow down the entire industry’s development progress—why not do it?

Also, stop threatening people's jobs, that's misanthropic.

Re: We must pace the frontier

#833

Post by solidasparagus »

barrrrald wrote: post #733
cuuupid wrote: post #457

At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record, - no open weights - can’t use claude to research AI - train on everyone else’s IP and sell it back to them - 8 regulatory capture attempts and counting - so controlling they are the only US company blacklisted by the US government This is not effective altruism / rationalism gone wild, it’s just monopolisti…

I have many issues with Anthropic, but I will say that their actions are fully consistent with a group of people who earnestly believe that AI is extremely dangerous In fact, I would say that the Occam’s razor explanation is not that they are seeking regulatory capture, but that they earnestly believe in the x-risk, and that they are the most thoughtful and capable people to address it You may disagree, you may think…

I think they are genuine believers, but I think it's naive to not think that commercial pressure doesn't play a role in their positions, either explicitly or, perhaps more likely, subliminally.

The x-risk stance and commercial stance have evolved to be the same thing - Anthropic must win, and then everything else seems to work backwards from that. Can you believe it, the path they think is best for x-risk involves them becoming filthy rich. And threats to their commercial dominance like distillation get framed in a way to turn them into x-risk concerns.

Re: We must pace the frontier

#834

Post by adsharma »

skydhash wrote: post #828
adsharma wrote: post #817

Earlier quoted context omitted.

The gun comes with a manual on how to use it safely. I'm sure it has some complexities, but at the high level: > A gun is a metal tube that uses a tiny, controlled explosion to shoot a small piece of metal (called a bullet) forward at very high speed. If the gun doesn't work as intended, you can take it to a shop and someone can fix it so it works as designed. All I'm saying is AI should be designed the same way. Tre…

Learning ML, there was a high emphasis on the error part of things as most of the course was on minimizing errors. After ChatGPT, there is a weird anthropomorphization going on, where it's all about hallucinations, alignment and what not. We have something that is statistical in nature so there should never been any expectation of error-free results/actions. The value has always been about discerning trends or the co…

Statistical learned indexes can exist in explainable tech such as a database.

In 2017 Google was writing papers about it. Then something changed.

I don't think it was the tech. It was a realization around the power and societal impact.

Re: We must pace the frontier

#835

Post by homieg33 »

softwaredoug wrote: post #676
bpodgursky wrote: post #610

Earlier quoted context omitted.

Do you have any familiarity with Anthropic at all? At no point did they ever say open weights are a good idea. Their entire thesis is AI IS VERY DANGEROUS AND WE MUST DO IT RIGHT. You can hate it, but everything they do is consistent with this thesis, and everything they say is consistent with their actions! You just want them to want different things.

It reads like “AI is dangerous, only we should be allowed to make money from it”

To me it reads like whoever can solve alignment should make money.

Re: We must pace the frontier

#836

Post by nixon_why69 »

kristofferR wrote: post #788
MisterTea wrote: post #713

Earlier quoted context omitted.

I feel like all the closed model providers are milking it as they likely know open models on local hardware will one day eat their lunch. We all know it's not a matter of if but when. The company goes bankrupt, the hardware and property sold off, banks holding the bag.

Cloud models will always have massive benefits of scale. Caching is the simplest one to understand, cloud providers often reach a 90% cache hit rate, so hosting the same request locally on the exact same model on the same hardware is often way less efficient than on the cloud where a group of users generates a healthy cache.

KV cache is per conversation, I'm getting 100% hit rate on my single tenant local set up.

The benefits of scale are on the token generation side, you can batch rounds and generate tokens for multiple conversations per pass instead of just one token per pass.

Re: We must pace the frontier

#837

Post by nullbio »

nullbio wrote: post #188

>> Each frontier AI company commits to giving ongoing, employee-like access to a team of embedded third-party evaluators (such as METR). Someone needs to look into who is funding METR (mentioned in Dario the Book Burners post explicitly). Because they keep popping up now, with close ties to people neck-deep in the orchestrated doomer hysteria media campaign and Anthropic. Looks to be highly coordinated that they are…

Ah and would you look at that, 1 day before this post from Dario, one of his safety leads quits, gets pumped all over the media with another doomer social post, and announces he's going to work for METR.

So he's literally stacking the deck. I really hope the US govt is not blind to everything that is happening here.

Re: We must pace the frontier

#838

Post by taneq »

karlgkk wrote: post #805
FusionX wrote: post #782

Earlier quoted context omitted.

We're already seeing anti-AI sentiments, but the movement is still fringe with a vocal minority. However, that'll change soon without alignment. Without self-intervention, there will invariably be future incidents that can cause major economic impact, leaked private data, loss of life (directly/indirectly) etc. Once that happens, their social capital is wiped. It'll be an avalanche of lawsuits and overzealous regulat…

> but the movement is still fringe with a vocal minority It’s easy to say “fringe” but the average person seems to have a generally negative sentiment around AI. But I wouldn’t say they have a firm opinion yet

The general sentiment I’ve seen is certainly negative, and seems to be driven by the anti-AI-art echo chamber and by LLM slop flooding the internet wasting everyone’s energy.

A few more informed people are also a little concerned about the end of the world, but that’s approaching from so many directions that an AI uprising might not be the worst option…

Re: We must pace the frontier

#839

Post by anon291 »

adsharma wrote: post #501

So many words. But missing the one that matters the most: Explainability. AI slowdown is worth it only if it can be made more explainable. Changing the language we use to discuss it is a good first step. We need to stop using inside baseball terms like alignment and mechanistic interpretability. Replace them with explainable tech. Graph Databases, Causality, Shared semantic spaces. Previous writings on the topic (als…

As much as I love all these technologies, they don't have any direction that approaches the efficacy of a transformer

Re: We must pace the frontier

#840

Post by GeneralMayhem »

encyclopedism wrote: post #764
softwaredoug wrote: post #676

Earlier quoted context omitted.

It reads like “AI is dangerous, only we should be allowed to make money from it”

Exactly He doesn't need anyones permission to do so, go ahead no one is stopping you! If you feel so strongly about it, lead by example. Perhaps others will follow, maybe even China. Regardless backup your sentiment with actions!

There is a specific, unilateral action that he is taking described in the article.

And the "why are they developing AI if they think it's so dangerous" argument is neither new nor persuasive. They're developing it because (1) they think the potential benefits are as great as the potential risks, and they know that if they aren't one of the actors on the frontier (2) they won't be able to propose meaningful solutions and (3) their opinions won't be taken seriously. Amodei is in rooms with powerful people to propose these things because Anthropic is successfully developing frontier models. The heads of various NGOs and advocacy groups that are concerned with AI safety but not themselves working on those systems are... nowhere, writing pamphlets and blogs that not you nor I nor anyone in Congress will ever read.

Post reply on HN