Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

471–480 of 946 posts

Re: We must pace the frontier

#471

Post by RyanShook »

MachineMan wrote: post #436

Its not the model, it is the compute that is the bottleneck. Dario says one thing but does the other as he scales energy and compute. It is and has been known that it will be ai that will be blamed for a wipe, that will be coordinated by human actors. If he really wants to not hurt people, he would shut it down. This blog post is therefore a classic case of covering his behind.

At least Altman is somewhat open about his desire to take over the world. Dario says one thing and does another making it impossible to know his true motives.

Re: We must pace the frontier

#472

Post by visarga »

esafak wrote: post #399
visarga wrote: post #388

Earlier quoted context omitted.

Yes, and we need to make microprocessors that do good work, not hack others. /s

Why is it so hard for people to understand the concept of agency in 2026?? What did your CPU do without instruction? Did OpenAI tell its agents to hack the orgs it did? Come on, make a half sensible argument.

As long as anyone can put anything they want in the prompt box there is no safety. It's just a tool, the danger comes from what it is being used for.

Re: We must pace the frontier

#473

Post by caaqil »

Chance-Device wrote: post #306

I like the idea of pacing the frontier, but while we’re talking about restrictions, I like restrictions of another sort more. Namely, restricting the use of AI in corporate environments so that it does not destroy the economy as it advances. The chance of getting broad agreement on “pacing” is fairly low, meaning that all of this likely won’t happen and the race will continue. However, even in the unlikely event that…

> Don’t use AI to replace every job, from graphic designer to accountant to software developer. Place legal restrictions on (at least) large companies such that they may not use AI to replace human labor in this way.

AI is taking all of our jobs. We need to control the means, nay, the pace of production. We should build some kind of barrier, not of concrete and cement but also of the regulatory variety, to stop these agents from coming into our special white color cubicles. It's the only way.

Re: We must pace the frontier

#474

Post by MachineMan »

reticulates wrote: post #53
TheSisb2 wrote: post #38

I know the common take online is that this is Anthropic doing pre-IPO marketing. I don’t think it is. I think Dario is genuinely afraid of the inevitability of AI turning into internet slime mold: consuming the environment, turning it into its own playground, and eventually manipulating human culture along with it. So am I. That said, if slowing this down were possible, I think it would have happened by now. Maybe he…

> I think Dario is genuinely afraid of the inevitability If someone genuinely believes that an invention will bring about the end of the world as we know it while making him and his friends inconceivably rich, “hey guys let’s uh, stop?” is so profoundly naive that his think pieces aren’t worth the bits they’re written on. He’s either an idiot or an idiot, does it matter whether or not he genuinely believes this nonse…

You are cynical but not cynical enough. The idea of a rogue Ai gives plausible deniability when they can blame human hubris, rather than it being seen as a deliberate and calculated attack, the perfect cover story for a sinister scifi plot. Make it look like an accident ehh

Re: We must pace the frontier

#475

Post by nbulka »

Shouldn't the cybersecurity tests be done on an airgapped, company-internal intranet? A network meant to mock the internet. Do not "test in production" like OpenAI did.

I'm mulling over if that be a good SaaS product or not - something mixing the Internet Archive with Tor, and Cloudflare ... seems like here is the place to suggest it and have others poke holes at the idea.

Re: We must pace the frontier

#476

Post by fwn »

twoodfin wrote: post #455
fwn wrote: post #402

Earlier quoted context omitted.

The biggest AI risk, by far, is the concentration of power in a few companies, located in the US. There is a lot of fear marketing about our text generators turning into Terminator. But other than the centralization of power, such fears are largely fiction. (Actual fiction, stuff like ai2027.) And the labs know it: If Anthropic or OpenAI believed in their own narrative of being on the brink of world dominating superi…

I dunno, do you really want to bet that if you took a deliberately unaligned frontier model and asked it to permanently disable the US power grid, it would do a poor job of it? What odds would you take on this bet? I’m disturbingly close to 50/50.

If we focus too much on the world domination fiction, we really might, in a moment of lapsed attention, end up plugging essential infrastructure into the public internet, hoping some dream of global alignment might save us.

That's like giving a gun to a monkey and hoping the monkey is trained well. ..a frontier monkey though.

Re: We must pace the frontier

#477

Post by [deleted] »

neomantra wrote: post #375

[In 2013], "Agents" caused Knight Capital to lose $450M in 45 minutes [1]. Implemented by humans and effected by computers, in the end it was really because of two reasons: * multiple levels of inappropriate controls and unintended consequences in several complex systems * the inability, both politically and technically, to turn it off [1] https://www.sec.gov/files/litigation/admin/2013/34-70694.pdf EDIT: Comments in…

[deleted]

Re: We must pace the frontier

#478

Post by joshheitzman »

franticgecko3 wrote: post #419
dismalaf wrote: post #409

This is corporate speak for "LLMs have hit a wall". I mean, it's been obvious for a bit, lately nearly all gains have been from harnesses (or whatever you want to call all the non-LLM bits that make up a chatbot or agent). If Anthropic and OpenAI were still seeing exponential or even linear gains from scaling they'd be doing it because the rewards to reaching AGI or SGI before everyone else are basically infinite. If…

That's just so extremely difficult to believe. Before December 2025 they were still intelligent code autocomplete or Stack Overflow bots, then they started one-shotting serious long horizon tasks. Now they've just solved a millennium prize problem. In less than a year.

> Before December 2025 they were still intelligent code autocomplete or Stack Overflow bots

This is false. Coding agents have been usable since at least May of 2025. I can't speak to earlier than that as May last year was when I personally started using them.

Re: We must pace the frontier

#479

Post by 6thbit »

If they intend to keep training then they can have, privately, increasingly advanced models that never become public because they don’t meet their “pacing requirements”.

Then they all build a larger moat cause china doesn’t get to distill private models for a while.

As long as the consumers “buy” this pacing and keep paying for current level public models, doesn’t seem risky from a business standpoint.

Alas, that’d also leave us gov in a position to seize the private models at any time.

Re: We must pace the frontier

#480

Post by alex43578 »

Chance-Device wrote: post #306

I like the idea of pacing the frontier, but while we’re talking about restrictions, I like restrictions of another sort more. Namely, restricting the use of AI in corporate environments so that it does not destroy the economy as it advances. The chance of getting broad agreement on “pacing” is fairly low, meaning that all of this likely won’t happen and the race will continue. However, even in the unlikely event that…

You just invented the job equivalent of rent control, and all the problems that ensue from it.
Post reply on HN