Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

551–560 of 935 posts

Re: We must pace the frontier

#551

Post by estearum »

andy99 wrote: post #544

How much of the “danger” is from better models vs the harness? Isn’t the current risk due to how AI is configured, like giving it a full set of tools and internet access and a goal to hack stuff? If we think we need laws or gate keeping, why isn’t it at this level? I already can’t ddos someone or fuzz their server or whatever right, I imagine if I threw equivalent compute at old school hacking I’d just get arrested.…

What in this proposal gives you the idea that they're only talking about pausing research itself? It doesn't say that at all.

But yeah, directionally I think you're right. It's insane we ever let these things connect to the Internet or to write, never mind execute code.

But for the economic counterargument, the distinction doesn't matter much. Continuing research but constraining harnesses is just accepting nearly all of the costs and foregoing nearly all of the benefits.

Who has appetite for that?

Re: We must pace the frontier

#552

Post by Loquebantur »

hex4def6 wrote: post #534
Loquebantur wrote: post #434

Earlier quoted context omitted.

You start from incorrect premises by presupposing agreed-upon rules would never work anyway. "Competition" only makes sense within a pre-mediated set of constrains. Rules all competitors agree to. Workable rules are a difficult problem, that doesn't mean it wasn't worthwhile to come up with them.

The greater the competitive advantage, the greater the incentive to "cheat" the rules. I am highly skeptical everyone will play by the rules, even if you could get people to agree on paper. The difference between something like "unlicensed AI training" and, e.g, Nuclear weapon non-proliferation is that nukes are realistically only a deterrence against invasion. There are no economic benefits to having them in a stock…

People don't "play by the rules" for uniform reasons: some have rational insight, others can be enticed and still others need to be shown averse consequences for breaking them. Rules aren't beneficial only when you hit 100% conformity, the mere possibility of them getting broken is no rational reason to abandon rules altogether.

"Alignment" likely is unsolvable programmatically: Conscious intelligence is more powerful than unconscious. How "aligned" are you? Would you want to be? What difference of any relevance is there between you and "artificial" conscious intelligence that allows you to turn them into slaves?

Re: We must pace the frontier

#553

Post by KaiserPro »

Aurornis wrote: post #397
Chance-Device wrote: post #306

I like the idea of pacing the frontier, but while we’re talking about restrictions, I like restrictions of another sort more. Namely, restricting the use of AI in corporate environments so that it does not destroy the economy as it advances. The chance of getting broad agreement on “pacing” is fairly low, meaning that all of this likely won’t happen and the race will continue. However, even in the unlikely event that…

> Namely, restricting the use of AI in corporate environments so that it does not destroy the economy as it advances. Regulate what, exactly? Limit what AI corporations can use? Other countries would love that more than anything. Basically a free gift to any competitors or any startup that quietly avoids the rules. There’s a theoretical version where all the countries in the world join hands and agree not to compete…

> world join hands and agree not to compete with each other

We kinda do for other areas.

Largely the world agrees not to pirate software, tv, other IP.

THe issue of joblessness is well understood in china, they know that jobless people means a drop in living standards, a drop in living standards means the "contract" has failed.

So it makes sense that countries like america and the constituents of the EU understand that loosing 10% of all jobs in a few years is going to be a massive dick punch.

I also I think that finally the historians and economic types havae got through to the dipshit billionaire that they only have money because the plebs are spending money. If the plebs are unemployed and unemployable, they (the billionaires) are going to be lynched.

Re: We must pace the frontier

#554

Post by hgoel »

RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

Why are we accepting the framing that the LLMs are felony generators, when the only incidences of LLM generated felonies involved misconfigured sandboxes and reckless waste of resources?

The companies doing these things without following common sense security measures are the felony generators.

Re: We must pace the frontier

#555

Post by devindotcom »

dramamine wrote: post #546
RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

[flagged]

throwaway bigot account

Re: We must pace the frontier

#556

Post by [deleted] »

tcdent wrote: post #204
xg15 wrote: post #118

I don't see how the dual goals of "we have to make an agreement with China for mutual slowdown" but also "we have to ensure we will always stay ahead of China" would work. I imagine the first thing the chinese government would demand in negotiations about such an agreement would be a lifting of the chip and distillation ban. > Some may believe these measures make it more difficult to cooperate with China, but I belie…

This is the part I find being left surprisingly hand-wavey. If you extrapolate it out, you see that it has a high likelihood of inciting physical force (read: military action) as a means of enforcement, so perhaps that's why nobody promoting ideals has been direct about it.

[deleted]

Re: We must pace the frontier

#557

Post by atombender »

academia_hack wrote: post #90

Dario's proposed approach is a classic example of capital attempting to control technological advancement and the means of production. For the first time in human history, any member of the working class can just about afford to have a team of expert scientist/physician/lawyer/engineers working directly for them. Super intelligence (the ability to have many smarter minds than your own reporting to you) has always bee…

If true AGI superintelligence is invented, I don't see how it will be made available to the masses, at least not intentionally. There will be no incentive for the likes of Anthropic and OpenAI to give such immense power to anyone else. Whoever has the superintelligence will be able to race ahead of everyone else.

However, such technology may be leaked, or it may (as what happened with LLMs) be so simple to replicate that anyone can do it. That doesn't mean it will; for example, LLMs are possible thanks to the affordability of GPUs, but they're only affordable right now (and increasingly less so) because of free market economics.

We are in the honeymoon phase of some tech that's in its infancy. The "democratizing" aspect of AI won't realistically last, I think. The current non-AGI AI won't necessarily go away, but it will be rendered obsolete.

Re: We must pace the frontier

#558

Post by estearum »

xg15 wrote: post #439
jfreds wrote: post #400

Earlier quoted context omitted.

I think taxes like these actually worsen the incentive structure. If it’s effectively more expensive to do training and inference, labs will be incentivized even more to improve efficiency. Opaque recurrence seems like a great way to improve efficiency, and it comes at the cost of safety

Even more opaque than the models already are? I take that risk. I also love how the labs are apparently screaming "Stop us! Please stop us!!!" at the top of their lungs while completely unable to escape their own incentive structure...

> I also love how the labs are apparently screaming "Stop us! Please stop us!!!" at the top of their lungs while completely unable to escape their own incentive structure...

It seems like you're implying there's some better alternative, but you're just describing a race-to-the-bottom. It is completely reasonable for every competitor to want an external coordinator (i.e. regulator) to break the pathological competitive dynamics.

Post reply on HN