Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

391–400 of 929 posts

Re: We must pace the frontier

#391

Post by csomar »

tcdent wrote: post #204
xg15 wrote: post #118

I don't see how the dual goals of "we have to make an agreement with China for mutual slowdown" but also "we have to ensure we will always stay ahead of China" would work. I imagine the first thing the chinese government would demand in negotiations about such an agreement would be a lifting of the chip and distillation ban. > Some may believe these measures make it more difficult to cooperate with China, but I belie…

This is the part I find being left surprisingly hand-wavey. If you extrapolate it out, you see that it has a high likelihood of inciting physical force (read: military action) as a means of enforcement, so perhaps that's why nobody promoting ideals has been direct about it.

Military action against whom? the US can’t even handle Iran let alone China.

Re: We must pace the frontier

#392

Post by stratos123 »

Ydarbleoj wrote: post #331
stratos123 wrote: post #252

Earlier quoted context omitted.

> disingenuous because if he believed what he says then he'd stop. Why do you think so?

Because I find the cognitive dissonance of saying it's dangerous while flying as fast as possible to an IPO too much to take. I just don't understand how others parse this differently.

So you're saying that if Dario truly thought what he says he does, he'd resign/dissolve the company somehow/unilaterally slow down? I see two problems with this:

First, Dario thinks that OpenAI will continue racing even if he stops, so there is, by his values, no point in stopping. Do you disagree with this, and think he should stop anyway?

Second, what's your take on, for example, Jacob Coxon, who did resign from Anthropic a few days ago after realizing they aren't being responsible? Do you agree that he, at least, does believe in AI risk?

Re: We must pace the frontier

#393

Post by polytely »

cja wrote: post #227

Can the frontier be paced partly by holding humans responsible for the actions of their software? My impression is that AI hacking is being treated as a special case where the AI itself is imagined to be responsible and the humans who created it, set it up and then ran it are somehow excused. I'm not a lawyer but surely the bad actions of AI are covered by existing law. I suspect that the development of AI would dece…

Remember that Chinese CEO that was executed for corruption & selling tainted baby milk powder [1]

I think these guys would improve their behavior if their actual life was on the line instead of that only being true in their less-wrong thought experiments.

1: https://en.wikipedia.org/wiki/Zheng_Xiaoyu

Re: We must pace the frontier

#394

Post by politician »

esafak wrote: post #382
rickydroll wrote: post #372

I also like the idea of pacing the frontier and slowing down development so we all have a chance to stop and catch our breath. However, I don't think regulating AI is the way to do it. I would tackle the problem by constraining the resources used to run AI models; the "simplest" way to do that is increasing the costs of running a data center. The two easiest places to raise data center costs are electricity and water…

This makes no sense. The hardware is only going to get cheaper. In a decade, everybody could be running an ASI on their phones. What we need to do is to make unaligned AIs illegal and monitor for them, like we do with nuclear weapons. And make aligned AI strong enough to counter it, for deterrence and defense.

I downvoted you because I think the OPs model is far simpler to implement and suffers from fewer conflicts of interest.

If we go down the regulation of alignment route, we'll have to ask experts to create those regulations and monitoring regimes. And which experts will the government ask? Oh, right, the parties that stand to benefit the most from regulation: OpenAI and Anthropic!

When "the experts" make certification cost $100M/model because "safety and alignment", then they will have cemented their duopoly.

Meanwhile, tariff rates on compute is a simple percentage that can be scaled up and down. Congress doesn't need experts to do that. Vendors can participate proportionally to their scale. This is far simpler and far more fair.

Re: We must pace the frontier

#396

Post by voidhorse »

maxutility wrote: post #179

I’m disappointed in the level of groupthink reflexive cynicism I see from commenters any time prominent AI leaders talk about AI risks and the need for regulation or pacing. Yes, regulatory capture is a risk, but this is also a profoundly unusual, fast moving, and potentially extraordinarily dangerous technology. There are strict regulations around nuclear weapons, as well as around US financial, energy, and other in…

This is a fine example of how rational good faith can quickly turn into naivety.

Consider the massive power and wealth differentials these companies stand to gain if they "win".

When the other participants have given up all reason and reasonable bounds on power, you'd do well to abandon any pretense of generous interpretation. Now is the time for critical analysis, not faith in the inherent "goodness" of someone whose entire job is presently to make as much money as possible by disrupting literally the entire white collar industry and who stands to gain much more than you or I do, and let's not forget, this outstanding success would never have happened without our work.

Keep in mind this is not a person who thought "wow, this is an existential threat, I'd better not contribute to building this". If anything Dario's past alarm ringing only proves that he's not really doing this in good faith. Otherwise he would have stopped a long time ago. The most generous interpretation of his behavior is that he literally thinks only anthropic is responsible and smart enough to build and control this stuff which...yeah, that should be pretty telling (esp. when you consider their recent security mishaps).

Re: We must pace the frontier

#397

Post by Aurornis »

Chance-Device wrote: post #306

I like the idea of pacing the frontier, but while we’re talking about restrictions, I like restrictions of another sort more. Namely, restricting the use of AI in corporate environments so that it does not destroy the economy as it advances. The chance of getting broad agreement on “pacing” is fairly low, meaning that all of this likely won’t happen and the race will continue. However, even in the unlikely event that…

> Namely, restricting the use of AI in corporate environments so that it does not destroy the economy as it advances.

Regulate what, exactly?

Limit what AI corporations can use? Other countries would love that more than anything. Basically a free gift to any competitors or any startup that quietly avoids the rules.

There’s a theoretical version where all the countries in the world join hands and agree not to compete with each other, but that’s so impractical that I don’t find it interesting to discuss.

> Place legal restrictions on (at least) large companies such that they may not use AI to replace human labor in this way.

So small companies get the advantage, but large companies don’t? And large companies in other countries also get the advantage?

This is a highly precise way for a country to take out all of their large companies. What would actually happen is that every large company would start the process of relocating to another country right away, accelerating job losses rather than slowing them.

Re: We must pace the frontier

#398

Post by joshheitzman »

kart23 wrote: post #214

> Do not sell powerful AI chips or semiconductor manufacturing equipment to China, and crack down on chip smuggling operations and remote access to data centers outside China. Chips will be the main determinant of China’s AI strength. > If we execute these measures well, I believe they would slow China’s progress enough to widen America’s lead significantly over the next 3–5 years — the window when AI becomes geopoli…

The lack of access to brute force their training seems to be resulting in them training more efficiently too such that they are quickly catching up despite current restrictions. Between that and the chip manufacturing capacity they are building I can't take this seriously.

Re: We must pace the frontier

#399

Post by esafak »

visarga wrote: post #388
esafak wrote: post #382

Earlier quoted context omitted.

This makes no sense. The hardware is only going to get cheaper. In a decade, everybody could be running an ASI on their phones. What we need to do is to make unaligned AIs illegal and monitor for them, like we do with nuclear weapons. And make aligned AI strong enough to counter it, for deterrence and defense.

Yes, and we need to make microprocessors that do good work, not hack others. /s

Why is it so hard for people to understand the concept of agency in 2026??

What did your CPU do without instruction? Did OpenAI tell its agents to hack the orgs it did? Come on, make a half sensible argument.

Re: We must pace the frontier

#400

Post by jfreds »

rickydroll wrote: post #372

I also like the idea of pacing the frontier and slowing down development so we all have a chance to stop and catch our breath. However, I don't think regulating AI is the way to do it. I would tackle the problem by constraining the resources used to run AI models; the "simplest" way to do that is increasing the costs of running a data center. The two easiest places to raise data center costs are electricity and water…

I think taxes like these actually worsen the incentive structure. If it’s effectively more expensive to do training and inference, labs will be incentivized even more to improve efficiency. Opaque recurrence seems like a great way to improve efficiency, and it comes at the cost of safety
Post reply on HN