Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

911–920 of 936 posts

Re: We must pace the frontier

#911

Post by vanviegen »

Applejinx wrote: post #562
vanviegen wrote: post #512

Earlier quoted context omitted.

Making balls ever rounder clearly has diminishing returns, as there's a limit to roundness. Is there a similar limit to intelligence though?

Yes, absolutely. Even human intelligence stops being useful at a certain point and becomes 'enough, provided the human is motivated and tenacious'. Intention is a better metric, but it's a serious AI weak point if not actively an achilles' heel. 'Warios' come to mind.

Intelligence in humans probably stops being useful for the individual at some level, indeed. But that might be due to being in an environment that's unable to feed that intelligence and execute on the ideas in generates.

Re: We must pace the frontier

#912

Post by ah1508 »

Centigonal wrote: post #495
ah1508 wrote: post #489

Don't you think that AI hate will limit the general use and then revenues so it will slow down by itself while the niche (AlphaFold for instance) will remains ? Origins of AI hate: * "my boss wants me to use AI but he does not understand my job nor how AI works" * "AI will kill all of us" * "AI will destroy my job (or my colleague's job if I use AI better than him)". * I cannot pay my electricity bills because of AI…

Banking on public sentiment to reduce adoption of a profitable technology could be dangerous. There's also a lot of hate for fossil fuels, gambling, health insurance, etc.

You're right, but fossil fuels industry were well established when people started to criticize them, you can even be against fossil fuels (I am) but locked in a system when your job is 30 miles away from your home without public transports. Even your toothbrush uses fossil fuels. A massive infrastructure has been built around fossil fuels, not easy to dismantle.

For AI it is different, it is just the beginning, leading companies are not even profitable and may never be if the majority hate this technology. VC funding will not last forever and corporate customers will not spend without limit like VC did.

I am not saying that "people are always right", but if an unexpected majority of people hates it, it does not help to keep a business running.

Re: We must pace the frontier

#913

Post by crossroadsguy »

FusionX wrote: post #782
RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

We're already seeing anti-AI sentiments, but the movement is still fringe with a vocal minority. However, that'll change soon without alignment. Without self-intervention, there will invariably be future incidents that can cause major economic impact, leaked private data, loss of life (directly/indirectly) etc. Once that happens, their social capital is wiped. It'll be an avalanche of lawsuits and overzealous regulat…

I am seeing a clear divide where people see their livelihood being adversely affected by AI.

Re: We must pace the frontier

#915

Post by googlybooglyboo »

A few thoughts: If things are so dire, it feels insufficient to just "slow down" and just "do a better job".

Why not concurrently publish or commit to publishing an initial audit? How is anyone to judge if this is the right move as opposed to being overly cautious, prudent, or reckless?

Why no concrete proposals about when or how such a pause could end? Obviously incredibly hard if not impossible, but it seems like securing critical infrastructure, web infra, consumer devices, etc would be a prerequisite. Or about how models have to be deployed so as to prevent breaking out of containment?

All in all, for an open letter, without digging into potential motivations or who benefits, it leaves me with a lot of questions.

Re: We must pace the frontier

#916

Post by robomartin »

cruffle_duffle wrote: post #908
robomartin wrote: post #447

My position on the clear gaslighting coming out of Anthropic is simple. 1: It is ridiculous on the face of it; it does not pass the physics test. They are actually claiming that AI will kill ALL OF HUMANITY (>10% probability) and have not provided a detailed explanation of exactly how they see that happening. Not one. Publish a paper showing exactly how you kill eight billion people while we all sit around and do not…

It’s the same thing with Covid. People thought 1 in 10 people would be dead. And believed it for years, despite it being orders and orders of magnitude wrong. Humans love a good end-of-world story. Always did, always will. It’s when they want the world to conform to their ungrounded irrational bat-shit crazy nonsense that I get a worried. Especially when people in power actually take their nonsense seriously.

Anthropic needs to shut down. They are promoting insanity from every pore for some crazy reason.

The best message these people can get from the public is to have their IPO fail miserably. Nobody should buy one share.

Either they are crazy, in which case you should not give them your money.

Or, they are truly building a doomsday machine, in which case you should not give them your money.

Ergo: Don't buy their IPO.

I've been trying to steel-man the argument and find a way in which AI could kill all of humanity. Maybe I am too stupid to come up with a clever enough idea. You would have to assume that eight billion people are NPCs in the game of life and simply fall over without any thought, effort, action or desire for self preservation.

Poof! Game over.

Right.

Re: We must pace the frontier

#917

Post by fragmede »

RobertDeNiro wrote: post #866
aubanel wrote: post #777

Earlier quoted context omitted.

You can disagree with Anthropic leadership, but all the points you mentioned can reconcile very well with them thinking in good faith "advanced AI is too dangerous to be left in all hands" Except the "train on everyone else IP" which can be said of all AI companies.

Problem with Anthropic is that they benefited massively from open source and now are completely against it. It’s hypocrisy at its worst. Why do they get to benefit, but not everyone else?

Because they got there first. It's not fair but life's not fair, and the same argument holds for use of fossil fuels and the USA's use of them vs China's. In Global Climate Change's sake, we only have the one planet so yes it's hypocritical, but if China fucks it up for everyone else on the planet, under the reasoning that the USA got to do it, so why can't we?

We're all dead!

Re: We must pace the frontier

#918

Post by boshalfoshal »

eknkc wrote: post #909
boshalfoshal wrote: post #803

Earlier quoted context omitted.

This is just completely false lol. They most certainly can make better models - why cant people here just read this for face value? I think anthropic is legitimately concerned about the safety implications of stronger models. Race dynamics necessitate that they make better and better models, which they clearly think is bad. They have an internal model which could solve a millenium problem end to end with no intervent…

> which they clearly think is bad These are companies with huge amounts of money invested in them. Their main responsibility is to their investors. If building better models (or making more money from the models they already have) would increase returns, they would do it. So they clearly can’t do it right now. Climate change hasn’t stopped the oil industry, and concerns about forever chemicals haven’t stopped the che…

I think you fundamentally misunderstand what kind of company Anthropic is. I'm simply taking the path of least resistance when understanding what people at Anthropic have to say, and basically every public source points to Anthropic talking the talk as well as walking the walk with regards to AI safety and their ideas on AI governance.

Re the point on returning shareholder value, Anthropic is a PBC:

> Public Benefit Purpose: Under Delaware law, a PBC allows directors to legally weigh social and public good alongside maximizing profit for stockholders

Investors surely know this and Anthropic probably has pretty reasonable legal grounds to stand on if they chose to prioritize safety over profits.

Seems pretty clear to me that this is in earnest.

Re: We must pace the frontier

#920

Post by hhhAndrew »

Shower thought: If a rogue AI agent can spread across the internet, it might be regulated under the same ruleset as other self-replicating dangerous critters: biosecurity / quarantine type laws. E.g. Who is allowed to own some live smallpox/agents? Who is allowed to study smallpox/agents to make a better smallpox/agent? Who is allowed to carefully expose smallpox/agent to environments where it can thrive? Some people, potentially, are allowed, under some rather regulated conditions ... but not, like, 10,000s of employees of some tech giants.
Post reply on HN