We must pace the frontier • Hacker News Board

Re: We must pace the frontier

#631

Post by stevenhuang » Sat, Sep 12, 2026, 9:32 PM UTC

hgoel wrote: post #627

aswegs8 wrote: post #592

Earlier quoted context omitted.

Yeah but why shouldn't this be possible? We learned that we can already create artifical intelligence that surpasses human intelligence in some dimensions. There is no natural barrier here. The pace of this improvement would be debatable, but what speaks against the possibility of such accelerating self-improvement?

In the real world there aren't any true exponentials, everything eventually saturates as ultimately physics related constraints hit. You can only compress information so much, transfer it so quickly, you can only access resources at a certain speed, only so much energy is available, etc. AI ultimately has to live in this reality and face the corresponding limitations. These companies have already consumed much of the…

> Recursive self improvement is ultimately limited by everything else that cannot move at the speed of electricity.

I'm not sure what your point is. No one thought RSI would break the laws of physics.

Re: We must pace the frontier

#632

Post by bravetraveler » Sat, Sep 12, 2026, 9:32 PM UTC

Everyone has a pace until they get punched in the model

Re: We must pace the frontier

#633

Post by pessimizer » Sat, Sep 12, 2026, 9:34 PM UTC

makerofthings wrote: post #603

They don't care about any of that. They're looking for more regulatory capture, to keep smaller labs from catching up by raising the cost of play, and perhaps keeping chinese labs away.

Yes, this is literally just dumb PR. It's Cambridge Analytica claiming that they were controlling people's minds, and people just eating that up because it's part of their juvenile SF fantasy.

I'm lying, it's smart PR. They're going to get the people opposed to AI to give them a monopoly on AI, let it be forced it into every nook and cranny of their society, and let it be priced arbitrarily while the big labs collude on a minute by minute basis. They're selling the problem and also selling the solution, like the best capitalists. And their solution is that they need to be allowed more power in order to sell more of the problem.

edit: And just like the Cambridge Analytica PR, it's got a serious political angle to it; and it's politicians who are really being wooed. If you're taking money from the AI labs, you run as being against the irresponsibility of the AI labs, the biggest fish come out and beg to be regulated just like the social media companies. To show good will, they funnel enormous amounts of money into those politicians, and do media tours about how awesome and therefore scary their product is. And something something China terrorists

Re: We must pace the frontier

#634

Post by lelanthran » Sat, Sep 12, 2026, 9:35 PM UTC

switchbak wrote: post #237

nickysielicki wrote: post #157

Earlier quoted context omitted.

> The best models are owned by a few companies In terms of cost per task, the open weight Chinese models are winning by a long shot. So it depends on what you mean by the best model.

That’s only one metric. It doesn’t matter how cheap it is, if it continually fails to solve a problem. Yes they’re getting better. No, I don’t think this will be a panacea. If only for the fact that this is China (more specifically the CCP) after all - they’ve got their own plan, and altruism is not part of it.

TBH it doesn't need to completely solve a problem.

I can use GLM to do in 2 hours what would have previously taken me a week.[1] Switch to Astra or Fable might take that down to 1 hour, maybe.

The difference is not so big when you look at it that way.

==================

[1] Actually, no, but let's pretend for the sake of argument that SOTA models can 10x - 40x your productivity.

Re: We must pace the frontier

#635

Post by Amekedl » Sat, Sep 12, 2026, 9:36 PM UTC

RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

Agreed; and it really is not that deep.

Realistically; anyone paying for llm access (anthropic, openai, gemini), is getting their access, and a service provided billed by tokens, subscription, whatever.

All the efficiency gains, which publications like deepseek v4.1 flash seriously frontload like it is their most important topic to have accomplished improvements on without diminishing performance too much - now this is a thing anthropic and anyone else also cares about, but for different reasons.

American "providers" with closed models are setting their token pricing somewhat arbitrarily, which is fine: it means more profit, and pretraining and RL experimentation is super important and expensive.

They (closed model providers) have very likely super optimized inference too, just like deepseek, but it's not at all something that any customer really has to care about - they just want the service to be as cheap and great as possible.

Re: We must pace the frontier

#636

Post by paulcole » Sat, Sep 12, 2026, 9:37 PM UTC

Great press release w/ the goal of pulling the ladder up behind him.

Additionally if you know your models aren’t going to get that much better AND you want to IPO in the near future, this is exactly what you would say.

Re: We must pace the frontier

#637

Post by zugi » Sat, Sep 12, 2026, 9:37 PM UTC

glub wrote: post #302

Dario, Sam, and Elon are all on the same page on this. So it's either they truly think AI is going to kill us all, or there's some other motives at play here. I don't think these people could possibly agree on the color of the sky, so what could the other possible motives be, based on what we know? OpenAI / Anthropic models have largely stopped advancing - that's not a good look when you're pre-IPO and Chinese models…

Yet Musk consistently opposes AI regulation - https://www.yahoo.com/news/videos/elon-musk-criticizes-ai-re... - even though it might help him.

Re: We must pace the frontier

#638

Post by augment_me » Sat, Sep 12, 2026, 9:37 PM UTC

Chance-Device wrote: post #306

I like the idea of pacing the frontier, but while we’re talking about restrictions, I like restrictions of another sort more. Namely, restricting the use of AI in corporate environments so that it does not destroy the economy as it advances. The chance of getting broad agreement on “pacing” is fairly low, meaning that all of this likely won’t happen and the race will continue. However, even in the unlikely event that…

This is a call to end capitalism. Its completely impossible to do what you suggest with the incentives of capitalism.

Re: We must pace the frontier

#639

Post by lelanthran » Sat, Sep 12, 2026, 9:37 PM UTC

villish wrote: post #414

znnajdla wrote: post #304

Earlier quoted context omitted.

Have you actually used the latest Chinese models? Because, in my experience, they're not just cheaper, they're actually better in many cases. In one recent experiment I did a few days ago on a task that I need in production at scale at my company, Qwen 3.8 and GLM 5.3 Flash were not just cheaper, the results were significantly higher quality than GPT 5.6 Sol. Even if China doesn't continue to release this stuff for f…

> Qwen 3.8 and GLM 5.3 Flash were not just cheaper, the results were significantly higher quality How can they be higher quality than models they were distilled from?

> How can they be higher quality than models they were distilled from?

The word "distillation" is not specific to AIs, has been in use for 100s of years, and does not mean the same thing as "dilution".

It means "getting a more concentrated form of the original product".

Re: We must pace the frontier

#640

Post by foxglacier » Sat, Sep 12, 2026, 9:40 PM UTC

monsieurbanana wrote: post #612

baq wrote: post #600

Earlier quoted context omitted.

The idea that a few hundred apes with nothing but a bunch of rocks could one day land on the moon and come back to earth safely must’ve sounded ridiculous a hundred thousand years ago

I like this comment because at least it's honest in the timelines for AGI

It's not honest in AGI timelines (only biological ones). It just accidentally supports your unsubstantiated belief. Your belief isn't magically true because you're somehow able to see the future when others can't. You're just arrogant.