We must pace the frontier • Hacker News Board

Re: We must pace the frontier

#961

Post by tripzilch » Sun, Sep 13, 2026, 9:25 AM UTC

aswegs8 wrote: post #592

zozbot234 wrote: post #577

Earlier quoted context omitted.

The entire idea of RSI is completely speculative and unproven anyway - the whole underlying claim is that you could prompt a frontier model (at some unspecified level of smarts) to "think about ways to improve your own architecture" and this would then result in the model becoming infinitely smart ("superintelligent") via some sort of foolproof, unconstrained positive feedback. It's more of a science fictiony trope t…

Yeah but why shouldn't this be possible? We learned that we can already create artifical intelligence that surpasses human intelligence in some dimensions. There is no natural barrier here. The pace of this improvement would be debatable, but what speaks against the possibility of such accelerating self-improvement?

> We learned that we can already create artifical intelligence that surpasses human intelligence in some dimensions

yeah they're called calculators

Re: We must pace the frontier

#962

Post by supergirl » Sun, Sep 13, 2026, 9:30 AM UTC

like the AC/DC current wars when Edison tried to scare people into adopting DC instead of AC.

Re: We must pace the frontier

#963

Post by aqme28 » Sun, Sep 13, 2026, 9:34 AM UTC

It's weird that we have two parallel discussions and they're not meeting in the obvious middle.

OpenAI is out there hacking HF, RubyGems, etc...

Dario is talking about restricting the frontier, pausing development.

The problem is that "restricting the frontier" benefits no one as much as the frontier labs. But there are other ways to force alignment-- make companies liable for these felonies their agents commit.

Make them liable, and they might actually devote the necessary resources to fixing these problems, rather than throwing their hands up and just saying "Trust nobody buy us", which obviously no one does.

Re: We must pace the frontier

#964

Post by a_ba » Sun, Sep 13, 2026, 9:37 AM UTC

Maybe instead of creating more rules we should trim down our legal systems and re-instate some basic accountability. I‘m pretty sure we have sent people to prison for running a botnet before and slapped material fines on torrenting operations.

Re: We must pace the frontier

#965

Post by nektro » Sun, Sep 13, 2026, 9:38 AM UTC

nektro wrote: post #559

cat's outta the bag. the time to stop was 5 years ago.

i wrote this in anger because coordination feels unlikely; but if it can be pulled off that would be positive

Re: We must pace the frontier

#966

Post by bytesandbits » Sun, Sep 13, 2026, 9:41 AM UTC

This is just bait for regulatory capture. It’s the only way in which you can find alignment of Elon, Sam and Dario within the same 24h.

Re: We must pace the frontier

#967

Post by tripzilch » Sun, Sep 13, 2026, 9:43 AM UTC

keeda wrote: post #842

hgoel wrote: post #554

Earlier quoted context omitted.

Why are we accepting the framing that the LLMs are felony generators, when the only incidences of LLM generated felonies involved misconfigured sandboxes and reckless waste of resources? The companies doing these things without following common sense security measures are the felony generators.

As TFA calls out, these agents were not asked to do any of these things and yet they did, at a bonkers scale, within just this handful of companies you mention. Whether they had leeway to is secondary to the fact that they did. Heck, they exploited zero day flaws which by definition means they went beyond common sense security measures. And now these agents are already being deployed all over the world at an ever inc…

> How much of the world do you think follows "common sense security measures"?

Well clearly all of them, cause so far it's only been this handful of companies running a felony-generator connected to a terminal and compute resources.

It would really help in these discussions if people wouldn't randomly jump between what actually happened and is happening, and things they envision/expect to happen at some point in the future ...

> these agents were not asked to do any of these things

no but they were clearly fine tuned to.

> at a bonkers scale

I mean let's not get hyperbolic

> they exploited zero day flaws which by definition means they went beyond common sense security measures

that's really not true. lots of common sense security measures protect against "zero day" flaws, it's called "defense in depth", and it was very much lacking

Re: We must pace the frontier

#968

Post by Gigachad » Sun, Sep 13, 2026, 9:44 AM UTC

cuuupid wrote: post #457

At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record, - no open weights - can’t use claude to research AI - train on everyone else’s IP and sell it back to them - 8 regulatory capture attempts and counting - so controlling they are the only US company blacklisted by the US government This is not effective altruism / rationalism gone wild, it’s just monopolisti…

> so controlling they are the only US company blacklisted by the US government

Their control here was refusing to make fully automated killing machines. They simply required someone to have to press the button.