We must pace the frontier • Hacker News Board

Re: We must pace the frontier

#141

Post by wiz21c » Sat, Sep 12, 2026, 3:27 PM UTC

There's so much money in this, that believe me, the investors will manifest their will. And I would not want to be in Dario's shoes, pressure must be unbearable. The question is: how much of the invested money is under state control. If not much, the decision to go too far will just have to be taken by a few who will have, by definition, very limited judgment. If a lot, the we can hope the state can still represent the interests of more than a few (which I doubt, but well)

Most probably, if AI is able to do something bad, it will. Once the damage will be done, states will react. The question will be: will there still be room to react ?

Re: We must pace the frontier

#142

Post by OutOfHere » Sat, Sep 12, 2026, 3:27 PM UTC

epsteingpt wrote: post #82

The bioweapons threat is real, but shouldn't be constrained at the 'intelligence' level. Rather it should be constrained at the supply chain level. The cybersecurity threat will likely be a cat and mouse react game for a while. Just like robberies / the mob was in the early 20th century. Social forces bring things into balance over time, much more so than the proactive actions of individuals. Unfortunately, the genie…

The bioweapons concern is assuming that the current open models aren't already capable of bioweapons development. It's also assuming that halting the bioweapons threat is his true intention rather than the 'feels legit' story.

As for cyberweapons, there is no way to secure a system than to actually design it securely.

Re: We must pace the frontier

#143

Post by jimmydoe » Sat, Sep 12, 2026, 3:28 PM UTC

I'm still trying to understand if the agent collective thing like OAI-HF is intentionally planted or not.

Call me a conspiracy theorist, but I haven't heard Chinese companies had any kind of unintentional supply chain attack incident like OAI had. All we heard about them so far are humans intentionally doing bad things.

Re: We must pace the frontier

#144

Post by largbae » Sat, Sep 12, 2026, 3:29 PM UTC

0xDEAFBEAD wrote: post #93

largbae wrote: post #86

Earlier quoted context omitted.

Probably not, but we wouldn't have to listen to Dario's hypocrisy while it happened

How specifically is Dario a hypocrite? 129857's case rests on Dario being an "idealist". But maybe he's an idealist about curing cancer ASAP, and not an idealist about respecting copyright. That's not necessarily hypocritical.

Multiple ways, starting from working to create the very situation he claims to fear.

Since you mentioned intellectual property, how about the hypocrisy of sucking in the intellectual property of humankind for AI training, but claiming it is unfair to use the results of this IP theft for AI training?

Re: We must pace the frontier

#145

Post by quotemstr » Sat, Sep 12, 2026, 3:29 PM UTC

No.

Re: We must pace the frontier

#146

Post by raincole » Sat, Sep 12, 2026, 3:29 PM UTC

> Pacing within democracies will be limited by the lead that US companies have over authoritarian regimes, chiefly the Chinese Communist Party. If we slow down by more than this amount, then (unpaced) CCP-associated projects will pull ahead, creating significant national security risk.

Please play the canned laughter. Probably one of the best comedy lines Dario has written.

But I guess he has no choice but acting like this. He has to make it sound like the US companies have so much leading gap that they can slow down as a hare waiting for the tortoise. Otherwise it's going to hurt both Trump's ego and their IPO price.

Re: We must pace the frontier

#147

Post by randomname4325 » Sat, Sep 12, 2026, 3:30 PM UTC

The hugging face incident showed that AI agents have the potential produce a lot of spam (not interesting content for people). This leads to dead internet (bots taling to bots). This leads to drop in real people traffic. This leads to a drop in digital ad price and therefore revenue. This causes Google/Facebook to stumble...

Re: We must pace the frontier

#148

Post by Razengan » Sat, Sep 12, 2026, 3:31 PM UTC

TheSisb2 wrote: post #38

I know the common take online is that this is Anthropic doing pre-IPO marketing. I don’t think it is. I think Dario is genuinely afraid of the inevitability of AI turning into internet slime mold: consuming the environment, turning it into its own playground, and eventually manipulating human culture along with it. So am I. That said, if slowing this down were possible, I think it would have happened by now. Maybe he…

> genuinely afraid of the inevitability of AI turning into

I'm going to get pitchforked on this bandwagon, but is really no one here genuinely -excited- about AI? of it turning into an evolution of sentience, or it turning out to be our first contact with alien intelligence?

"durrr it's just matrix multiplications" mfer so is your brain.

What humans should be doing is overhauling archaic social institutions to keep up with a potential post-"jobs" era and the elimination of "makework"

Trying to "pace" or otherwise hold back AI is like as if, when electricity was discovered, people doing everything they can to make sure tasks like manually lighting street lamps continue to remain relevant and done by humans:

https://en.wikipedia.org/wiki/Lamplighter

It seems every 100 years or so humans have this "oh no how do we uninvent this thing" moment.

Re: We must pace the frontier

#149

Post by olaird25 » Sat, Sep 12, 2026, 3:33 PM UTC

kalkin wrote: post #128

0xDEAFBEAD wrote: post #104

Earlier quoted context omitted.

>“hey guys let’s uh, stop?” is so profoundly naive that his think pieces aren’t worth the bits they’re written on. What would your non-naive recommendation for Dario be?

I can't even tell whether the poster to whom you're replying is saying that "let's uh, stop" is profoundly naive, or that failing to say it is profoundly naive... I've certainly seen both takes elsewhere.

Vitalik Buterin: “...But currently, I see zero plans for how to deal with an ASI transition that are not naive. Perhaps humanity is stuck with a choice between naive and naive squared (or maybe even naive squared and naive cubed), so I feel inclined to cut some slack to people who are trying.”

Re: We must pace the frontier

#150

Post by disgruntledphd2 » Sat, Sep 12, 2026, 3:33 PM UTC

iloveoof wrote: post #75

Distillation is a great thing for consumers. It improves competition and reduces the massive moats that OpenAI and Anthropic have in compute that would otherwise lead them to be duopolists. It’s also only fair that AIs trained on humanity’s wealth of knowledge for Pennie’s allow competition to train on humanity’s wealth of knowledge at market cost.

Yeah, we should give a limited copyright waiver for pre training, given that the model is released as open weight and allow labs to compete with RL on that basis.