Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

241–250 of 941 posts

Re: We must pace the frontier

#241

Post by nullbio »

cja wrote: post #227

Can the frontier be paced partly by holding humans responsible for the actions of their software? My impression is that AI hacking is being treated as a special case where the AI itself is imagined to be responsible and the humans who created it, set it up and then ran it are somehow excused. I'm not a lawyer but surely the bad actions of AI are covered by existing law. I suspect that the development of AI would dece…

That's the -only- way it should be paced. But then regulatory capture wouldn't happen, so of course they won't suggest this one.

Re: We must pace the frontier

#242

Post by switchbak »

academia_hack wrote: post #131
angusturner wrote: post #115

Earlier quoted context omitted.

This seems highly optimistic to me. What's to say current AI won't be highly power-concentrating by default? The best models are owned by a few companies, and displacement of knowledge-workers mainly seems to benefit the capital class. On-device / edge computing makes sense in a few very limited scenarios. And economically, price or watt/token (or watt/task completed) might always be better in large data centers. No…

I'm not terribly optimistic. I don't think humans have really figured out what to do about historical materialism. More just remarking that this is a blog post that almost literally reads "fellow elites, let us ensure only we control the means of production. It is our duty to humanity to control the masses."

I’m trying to hear what you’re saying, but using these Marxist cliches makes your statements a little off putting to me.

Re: We must pace the frontier

#243

Post by nullbio »

heaney-555 wrote: post #11

None of this works without buy-in from China. This isn't something private companies can decide. The US would need to sign a groundbreaking deal with China, equivalent to the Anti-Ballistic Missile Treaty of the Cold War.

When I was reading this I was chuckling to myself imagining how China would be interpreting it as they read it. It was something like: Fuck you.

I hope China tells him to go kick rocks. From everything that has happened, they are the only ones carrying the torch for humanity that have led us to having some semblance of a healthy open-source/open-weight ecosystem.

No one in China is carrying on like headless chickens about the world ending, either. They're rational pragmatists, getting things done.

Re: We must pace the frontier

#244

Post by streptomycin »

spidersouris wrote: post #52
TheSisb2 wrote: post #38

I know the common take online is that this is Anthropic doing pre-IPO marketing. I don’t think it is. I think Dario is genuinely afraid of the inevitability of AI turning into internet slime mold: consuming the environment, turning it into its own playground, and eventually manipulating human culture along with it. So am I. That said, if slowing this down were possible, I think it would have happened by now. Maybe he…

The more I read about everything that has been written regarding AI regulation since the OAI/HF incident, and the more it reminds me of the nuclear arms race (although the potential consequences would possibly be very different). Surely, we managed to negotiate a Non-Proliferation Treaty with most of the countries of the globe. Cannot we take inspiration from that for AI?

Indeed, as Dario wrote in this post:

> This could be seen as analogous to the SALT treaties — capping the number of missiles limited the potential for destruction while preserving each country’s deterrent

Re: We must pace the frontier

#245

Post by dainiusse »

mofeien wrote: post #73
dainiusse wrote: post #56

It is always like that. Whenever anthropic hits some wall (like now beaing beaten by astra) - there comes some article like that. Seen that enough times.

Yesterday the same arguments were used about OpenAI when Sam Altman said something similar: https://news.ycombinator.com/item?id=49652270 It gets to the point that by Occam's Razor the more likely and reasonable explanation is that Dario Amodei and Sam Altman are just actually afraid of the disastrous impact ASI may have on the world, and that the race they're in is not good, and calling for help to governments in fo…

Or they want a wall against competitors. Who knows.

Re: We must pace the frontier

#246

Post by [deleted] »

Jcampuzano2 wrote: post #110
0xDEAFBEAD wrote: post #89

Earlier quoted context omitted.

Aren't they beholden to investors already? Why would an IPO make a big difference? In any case, if they need to IPO in order to survive as a company, then bringing in regulators to act as a counterweight against commercial pressures makes a lot of sense to me.

If anything, his entire argument leads to the eventual premise that AI labs need to nationalize. He is already arguing that commercial competition creates dangerous incentives, and that labs cannot slow down because competitors may overtake them. He also asks for what would boil down to significant government intervention to preserve a Western lead. If this is true, why preserve the commercial incentive? The U.S. gov…

[deleted]

Re: We must pace the frontier

#247

Post by glub »

shepherdjerred wrote: post #213
glub wrote: post #138

Earlier quoted context omitted.

And all of this information has been available on the open web way before LLMs. I'd wager it's likely easier for an average person to do this with Tor browser than it is to get an LLM to help them with it. Even ones that Dario calls dangerous.

You are right that it’s easier to do this via Tor than an LLM. Those are the safeguards…

So what's the fuss about then? Is the idea that a Chinese company will release a model that will have no safeguards? For what purpose?

Basic safeguards are all that's required, and they've been there in every usable model since GPT-2, including Chinese models that are supposedly "unsafe".

Or are we saying that some lunatics will start training their own models, spin up a GPU cluster, run some abliteration workflow, or learn how to jailbreak?

That would be a very dedicated person. And dedicated person doesn't need an LLM. So where are they?

Re: We must pace the frontier

#248

Post by nullbio »

HarHarVeryFunny wrote: post #101

Somewhere along the way Amodei seems to have become corrupted by power and/or impending extreme wealth. In early interviews with people like Dwarkesh he's the likable geek gushing about scaling laws, animated and able to maintain eye contact with the interviewer. He is now a different person - a political manipulator with a bizarre unsettled interview demeanor avoiding eye contact and looking from side to side. Props…

> In early interviews with people like Dwarkesh he's the likable geek gushing about scaling laws, animated and able to maintain eye contact with the interviewer. He is now a different person - a political manipulator with a bizarre unsettled interview demeanor avoiding eye contact and looking from side to side.

I noticed this too, as soon as he started going on the press cycle, following in the footsteps of Sam Altman. It all has gotten to his head, he sees himself as some sort of God now.

Re: We must pace the frontier

#249

Post by stratos123 »

glub wrote: post #44

> We have sought a middle way: to show that it’s possible to build carefully and succeed commercially, and to make safety something on which AI companies compete. In other words, to create a race to the top > Transparency. Regardless of what commitments we make, the public deserves to know what is going on. Anthropic has been a supporter of transparency for a long time And then it goes on tangent of how we should pac…

> Suppose it was, I don't know, Australia producing SOTA open weights, because they believed it was the right thing to do for the benefits of humanity - would Dario propose making Australia a geopolitical adversary too?

I'd expect he thinks that people are capable of realizing that AI is very dangerous and also that democracies end up mostly representing the will of the people, from which it follows that Hypothetical AI Leader Australia would agree to ban it too. This argument doesn't work for countries which don't care what their citizens want, like China.

I do think that it's a questionable decision to alienate China this much in this essay, instead of leaving open the possibility of China agreeing to a treaty that'll limit their progress. I suspect Dario is doing this to signal his allegiance with the US government, in hopes to increase the chance they'll go along with him, which is an unfortunate choice but plausibly the correct one.

Re: We must pace the frontier

#250

Post by 0xDEAFBEAD »

nullbio wrote: post #239
PunchyHamster wrote: post #88

Earlier quoted context omitted.

I dunno man, it all just sounds like trying to put controls on AI while being the favourite child of govt so competition can't fight as easily. Especially with IPO around the corner

And getting to choose his own "embedded evaluator" org that has deep ties to everyone in the doomer media campaign. It's all so obvious.

Who would you pick for "embedded evaluator"?

"deep ties to everyone in the doomer media campaign"

Are you suggesting these external NGOs were spun up as part of a gigantic pre-IPO hype stunt? METR was founded multiple years ago. This "stunt" is getting quite elaborate.

https://substackcdn.com/image/fetch/$s_!O0R5!,f_auto,q_auto:...

At a certain point, Occam's Razor says: These engineers are legitimately worried. They haven't invested years in a bizarre reverse psychology campaign to convince the public that their product is dangerous in order to make more money.

Post reply on HN