We must pace the frontier • Hacker News Board

Re: We must pace the frontier

#201

Post by sailfast » Sat, Sep 12, 2026, 4:11 PM UTC

ThrowawayR2 wrote: post #133

NiloCK wrote: post #45

Earlier quoted context omitted.

The degradation, polarization, and weaponization of or media landscape over the era of social media has left us utterly incapable of believing anything that anybody says. Dario in particular has consistently been risk-wary on model improvements for going on a decade - long before he was CEO of Anthropic. He believes what he is saying. Human beings can plainly say things that they think are true. Not every utterance b…

> " Human beings can plainly say things that they think are true. Not every utterance by every person is a cynical ploy. " Every person, no, but every utterance of a CEO is, in fact, a cynical ploy. That's not even a cynical statement itself; it is literally a core part of a CEO's responsibilities to represent the corporation they manage in a way that benefits the corporation. If he believes what he's saying, then wh…

Not sure you can judge a PBC under those same umbrella as a for profit company, and voting rights make a big difference in terms of responsibility. There are a number of ways to govern companies that can reduce the amount of cynicism and “shareholder value” issues - you just don’t see it very often.

Re: We must pace the frontier

#202

Post by dham » Sat, Sep 12, 2026, 4:11 PM UTC

We can cure Cancer. No wait nevermind, actually slow us down.

If we are actually close to ASI then no one in their right mind would say slow down

Re: We must pace the frontier

#203

Post by Ydarbleoj » Sat, Sep 12, 2026, 4:12 PM UTC

The more he talks the less seriously I take him.

Re: We must pace the frontier

#204

Post by tcdent » Sat, Sep 12, 2026, 4:12 PM UTC

xg15 wrote: post #118

I don't see how the dual goals of "we have to make an agreement with China for mutual slowdown" but also "we have to ensure we will always stay ahead of China" would work. I imagine the first thing the chinese government would demand in negotiations about such an agreement would be a lifting of the chip and distillation ban. > Some may believe these measures make it more difficult to cooperate with China, but I belie…

This is the part I find being left surprisingly hand-wavey.

If you extrapolate it out, you see that it has a high likelihood of inciting physical force (read: military action) as a means of enforcement, so perhaps that's why nobody promoting ideals has been direct about it.

Re: We must pace the frontier

#205

Post by stratos123 » Sat, Sep 12, 2026, 4:13 PM UTC

youoy wrote: post #176

0xDEAFBEAD wrote: post #124

Earlier quoted context omitted.

What course of action would you suggest for Dario at this point?

For example I believe the world would be more secure with an open frontier AI lab, but the consequences of him doing that are too big for him.

You mean, if Anthropic made all of their models open-weights from now on? I'm pretty sure Dario thinks this will be extremely unsafe, because it'd provide unrestricted access to all the most capable/dangerous models to everyone, and hence doesn't do it. Why do you think it'd make the world more secure, if he did?

Re: We must pace the frontier

#206

Post by RandomLensman » Sat, Sep 12, 2026, 4:13 PM UTC

bee_rider wrote: post #159

RandomLensman wrote: post #62

Why is the analogue necessarily the regulation and control of nuclear weapons (e.g., SALT) and not, for example, that of bioweapons? Some very different paths are available. On both I would note that the private sector has only a limited role, though.

I don’t think the ban of bioweapons has really been tested. Bioweapons have a lot of downsides that make them not very appealing to deploy anyway (high chance of getting your own team, not terribly fast moving, disciplined modern soldiers in top-tier militaries are usually willing to comply with health rules). It’s a ban against doing something basically stupid and ineffective for the most part. With “AI” technology,…

Related to not tested: Who has a large bioweapons arsenal? As per below, my point was more about that bans do happen.

Re: We must pace the frontier

#207

Post by esseph » Sat, Sep 12, 2026, 4:13 PM UTC

ActionHank wrote: post #66

yewenjie wrote: post #51

Earlier quoted context omitted.

Because he genuinely believes if he doesn't do it the next guy will do it worse.

Dude probably sleeps on a bed worth more than your networth. You have no frame of context to understand his intent or legitimate worries. The only applicable perspectives are to trust or apply logic. It is foolish to trust someone you don’t know who stands to benefit from lying to you. Logic dictates that given the ungodly sum of money he stands to gain, he will lie to everyone who will listen.

> Dude probably sleeps on a bed worth more than your networth.

I never understand shit like this coming out of people's mouths. Never.

It's not a judge of actual Worth as a human being, it's not a judge of capability or competence or ethics. It's not a judge of actual skill or ability. It doesn't make them a better cook, a better spouse, a better parent or lover. It doesn't make them more dangerous or more skilled at anything.

It makes them financially wealthy for at least a set period of time.

Cancer and time and 5.56mm still impact them the same way as every else.

It's Pharaoh worship psychology nonsense, and it's fucking embarassing to read.

Re: We must pace the frontier

#208

Post by narnarpapadaddy » Sat, Sep 12, 2026, 4:13 PM UTC

fasterik wrote: post #186

Jcampuzano2 wrote: post #110

Earlier quoted context omitted.

If anything, his entire argument leads to the eventual premise that AI labs need to nationalize. He is already arguing that commercial competition creates dangerous incentives, and that labs cannot slow down because competitors may overtake them. He also asks for what would boil down to significant government intervention to preserve a Western lead. If this is true, why preserve the commercial incentive? The U.S. gov…

I don't agree that the argument implies nationalization. Regulation would work if it slowed everyone down at the same rate, enough to mitigate the risks. The problem is that you need regulators who know what they're doing and strong international cooperation. If you regulate a fraction of the global market, you just create an incentive to shift development to other countries. Nationalization, being essentially the mo…

I think this is the correct take. And until we have an AI Hiroshima it’ll be difficult to get the international community to work together. It’ll take rogue AI (or AI-powered group) disabling a significant world power before everyone comes to the table. Otherwise, it just looks like MAD and the equilibrium holding to the powers that be.

Re: We must pace the frontier

#209

Post by ks2048 » Sat, Sep 12, 2026, 4:13 PM UTC

heaney-555 wrote: post #11

None of this works without buy-in from China. This isn't something private companies can decide. The US would need to sign a groundbreaking deal with China, equivalent to the Anti-Ballistic Missile Treaty of the Cold War.

Nothing says "Let's make a deal" like constantly insisting we are Good and they are Evil.

Re: We must pace the frontier

#210

Post by sailfast » Sat, Sep 12, 2026, 4:14 PM UTC

indoorfish wrote: post #25

Sevii wrote: post #13

Earlier quoted context omitted.

We'd have to make a deal with China and be confident they wouldn't cheat on it.

Would this be similar to the deal of "we'll offshore all our manufacturing to you and you'll become a free, open, liberal democracy with open borders and multiculturalism?" Because I remember how that deal turned out.

That last part was never important. Trading partners very rarely go to war. There just isn’t much in it for either party. Close cooperation and dependency is actually quite important here - liberal democracy or no.