Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

791–800 of 943 posts

Re: We must pace the frontier

#791

Post by pvab3 »

matheusmoreira wrote: post #711
malfist wrote: post #698

Earlier quoted context omitted.

If I set my walet beside me and you swipe in walking by, you have still committed theft. Victim blaming isn't legally acceptable

Nah. I'm definitely going to blame the people who built a trivially exploitable system and got rich off it while everyone else has to deal with the consequences. By the way, you didn't commit theft. It's more like credit card fraud. User just disputes the charge and it kind of disappears. The banking system just absorbs it, because the optimal amount of fraud is non-zero. https://www.bitsaboutmoney.com/archive/optima…

Not every system that's exploitable is the deliberate result of cut corners. If you threw enough compute at exploiting a Casio calculator you could get somewhere.

Re: We must pace the frontier

#792

Post by meander_water »

> I believe that AI could cure most major diseases in the next 5–10 years, greatly accelerate economic growth rates, create a world of abundance and empowerment, and usher in a renaissance of democracy and freedom

translation: I'm going to build a product and hope that it works, promise it to be a panacea while realistically having no control over how it's used, the wider economic impacts it causes or how it changes the mental health and cognition of its users.

Re: We must pace the frontier

#793

Post by sejje »

pants2 wrote: post #756
hgoel wrote: post #627

Earlier quoted context omitted.

In the real world there aren't any true exponentials, everything eventually saturates as ultimately physics related constraints hit. You can only compress information so much, transfer it so quickly, you can only access resources at a certain speed, only so much energy is available, etc. AI ultimately has to live in this reality and face the corresponding limitations. These companies have already consumed much of the…

We don't know exactly what the limits of AI improvement on our current infrastructure are, though. If the human brain is 20W, and a datacenter is 1GW, then maybe that datacenter can be 50 million times smarter than a human. If that's not already a risk to humankind I don't know what is.

Is the datacenter gonna grow legs?

Give me one datacenter, I'll keep it under control all by myself.

Now, if some dumbasses start hooking up their data centers to...I don't know, like--robot factories? That sounds like a risk to humankind.

Re: We must pace the frontier

#794

Post by CuriouslyC »

zozbot234 wrote: post #577
RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

The entire idea of RSI is completely speculative and unproven anyway - the whole underlying claim is that you could prompt a frontier model (at some unspecified level of smarts) to "think about ways to improve your own architecture" and this would then result in the model becoming infinitely smart ("superintelligent") via some sort of foolproof, unconstrained positive feedback. It's more of a science fictiony trope t…

That's not how it works. Look at AlphaEvolve. The model generates hypotheses and designs experiments, and the results of those experiments are fed into the next round, with notable results percolated up to humans for refinement.

Re: We must pace the frontier

#795

Post by chanakya »

FusionX wrote: post #782
RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

We're already seeing anti-AI sentiments, but the movement is still fringe with a vocal minority. However, that'll change soon without alignment. Without self-intervention, there will invariably be future incidents that can cause major economic impact, leaked private data, loss of life (directly/indirectly) etc. Once that happens, their social capital is wiped. It'll be an avalanche of lawsuits and overzealous regulat…

Exactly right. A slow down to enable deeper work on alignment is welcome, not matter what the motivations.

Re: We must pace the frontier

#796

Post by kilgarenone »

If we had practiced "precautionary principle", we wouldn't be in this yet another human predicament again. But atlas, the current rapacious hubristic techno-optimist competitive capitalistic system made that an non option in the first place.

Re: We must pace the frontier

#797

Post by afthonos »

throwaway7783 wrote: post #706
afthonos wrote: post #693

Earlier quoted context omitted.

You are incorrectly cynical. They are telling you things are bad, and because you refuse to countenance they could be worse, you assume they must be better to comply with your mandate to disbelieve. A true cynic looks at the statements by the AI labs, assumes things are worse because the labs want to seem better than they truly are. And it takes a special kind of mass delusion to drive a sane person to think “AI is c…

The cynicism is about motivations and not that they are inherently not bad. Perhaps they are as bad as they claim. Or perhaps they're worse. All we have a couple of run of the mill breach examples and some people inside the talking about how dangerous it is. Yes, they are far more qualified than I am (or most people here), and perhaps there is a grain of truth. It is the motivation - and it is always money with corpo…

It is always money — but it isn’t always only money. They are not asking for anything that will prevent them from making money in the future, but they are asking for help stopping the runaway train they’re on. These are compatible requests.

Re: We must pace the frontier

#798

Post by ozozozd »

cpeterso wrote: post #379
kennywinker wrote: post #323

Earlier quoted context omitted.

> As a non-US citizen, which authoritarian powers is he referring to? As far as I'm aware, the US is the only country using frontier models to conduct air strikes on civilians and orchestrate assassinations of foreign governments. Israel is doing that too. I believe Russia and Ukraine have both also used AI powered autonomous drones in warfare at this point - tho probably not frontier ones, since they need to run on…

> Iran and Houthi rebels used Anthropic's Claude AI to target US warships and build hypersonic missiles — Houthi rebels also used the bot to code ballistic missile guidance systems https://www.tomshardware.com/tech-industry/artificial-intell...

And the source for the claim does not even clear the bar for r/bodybuilding, because it’s “trust me bro.”

Re: We must pace the frontier

#800

Post by CuriouslyC »

nwienert wrote: post #751
matheusmoreira wrote: post #659

Earlier quoted context omitted.

I disagree. OpenAI's moat is their massive amounts of compute. They're providing an absurd amount of value with their subscriptions and resets. If anyone's dead in the water, it's Anthropic. Even Fable isn't enough anymore. This "safety" nonsense is the only play they have left, and nobody really cares about their fearmongering.

Anthropic gives you much more compute with their $200 plan, inclusive of resets, and this has been true for a very long time. There was only a brief window of time that the opposite was true.

I've used the $200 dollar Anthropic plan @ Opus4/4.1, 4.5 and 4.8, and the $200 OAI plan from GPT5-6, and at every point in time my anecdotal experience is that the OAI limits are FAR more generous. I could consistently burn my weekly limits in ~36h on Opus, but it's hard to do it in less than ~72h with GPT.
Post reply on HN