Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

511–520 of 942 posts

Re: We must pace the frontier

#511

Post by purplepintoss »

amanfromsolan wrote: post #96
jraines wrote: post #35

Frankly, I am sick of "it's all just marketing and attempt to do regulatory capture, and this is obvious to me, a smart person" on every single discussion related to safety. There could be an element to truth to it, but it's certainly not the entire story and is just so tiresome at this point.

Couldn’t agree more. I feel my heart sinking where anytime anyone suggests any regulation for a potentially, suggested by multiple people, world changing or destroying tech, all the moments here are “ah you billionaire, ha! trickster If you believed it, you would SHUT anthropic or release all models open source.” And then what? How does that help with slowing down the frontier? Should he shut shop and then grovel Sam…

This is circular logic.

You're saying nobody can doubt that it's world destroying because it's potentially world destroying.

> Anthropic is going to be worth a trillion

And we're done here.

Re: We must pace the frontier

#512

Post by vanviegen »

mullingitover wrote: post #385
angusturner wrote: post #115

Earlier quoted context omitted.

This seems highly optimistic to me. What's to say current AI won't be highly power-concentrating by default? The best models are owned by a few companies, and displacement of knowledge-workers mainly seems to benefit the capital class. On-device / edge computing makes sense in a few very limited scenarios. And economically, price or watt/token (or watt/task completed) might always be better in large data centers. No…

> The best models are owned by a few companies Whenever I see 'best model,' 'most advanced model,' etc, it just reads to me as 'roundest ball'. The new model is the most precisely round ball ever, models next year are going to be even rounder , etc. The difference in utility between the latest, most round ball and last year's frontier balls (which are now freely available to the public) is pretty debatable, imo.

Making balls ever rounder clearly has diminishing returns, as there's a limit to roundness.

Is there a similar limit to intelligence though?

Re: We must pace the frontier

#515

Post by esafak »

convolvatron wrote: post #415
politician wrote: post #394

Earlier quoted context omitted.

I downvoted you because I think the OPs model is far simpler to implement and suffers from fewer conflicts of interest. If we go down the regulation of alignment route, we'll have to ask experts to create those regulations and monitoring regimes. And which experts will the government ask? Oh, right, the parties that stand to benefit the most from regulation: OpenAI and Anthropic! When "the experts" make certification…

I think the point is even stronger. alignment is really not very well defined, nor are the mechanisms used to provide it. from what I understand it's more of a statistically imposed bias than an impenetrable gate. personally I would like to flip this around, all of this is a consequence of some really extreme notions about investment. its a tail-wagging-the-dog. the investment community is apparently in love with the…

Probabilistic safety is too low a bar. We need to project the model outputs onto a safe subspace; lobotomize them, if you will. It may make them dumber, but that's a fine price to pay.

I don't work in this space so I don't know the latest, but here's an example: Provably safe systems: the only path to controllable AGI (https://news.ycombinator.com/item?id=37619285)

Re: We must pace the frontier

#516

Post by 8note »

nunez wrote: post #169
129857 wrote: post #64

Earlier quoted context omitted.

MSFT is good, they said when it bought GitHub. MSFT is a reformed company and supports open source, they said. Then MSFT stole all IP from GitHub and made it worse and fired developers. Amodei does not care in the slightest about ruining software development and making people unemployed. Maybe he cares about bio-weapons because those could kill him, too. That is it. If he were an idealist, he wouldn't steal (literall…

Giving credit where credit is due, I believe Dario and his squad formed Anthropic because he and Altman couldn't align on safety. The only way to build models like the Claude series is to play dirty and train on LITERALLY ALL the data. Something something Pandora's Box Torment Nexus...

i dont think thats particularly worthy of credit?

somebody actually invested and trustworthy wouldnt be skirting people's rights to make a killer robot.

we havent written it down, but from how everyone reacts, you need permission to train and do inference based on somebody's work. its a right

Re: We must pace the frontier

#517

Post by SheinhardtWigCo »

> Any cooperation we are able to achieve with China will extend the amount of time we have to spend on pacing the frontier within the democratic nations.

Any cooperation we are able to achieve with China will extend the amount of time we have to secretly establish permanent AI (i.e. military) superiority.

China knows this, so the suggestion that they will play ball is absurd.

Re: We must pace the frontier

#518

Post by causal »

michaellee8 wrote: post #465
cardamomo wrote: post #450

Earlier quoted context omitted.

These do not seem to be mutually exclusive. Should we not do both?

Do you really think we can really get every country to truly pace the frontier? Pretty sure China won't give a f until they catch up Anthropic and OpenAI. It is an arm race. We had nukes for like 70 years and still haven't figured out how to make every single country follow those nuclear treaties, with an increasingly non-interventionist US I don't think we can get every single country to the table and agree to a pau…

Didn't answer the question...

Re: We must pace the frontier

#519

Post by HellDunkel »

thadt wrote: post #430

Hard disagree. Our choices are: A) Bet our collective good on the national and international cooperation of all companies, nations, and people to come together in order to slow down development of one of the most powerful economic tools (and/or weapons) the world has ever known. Or B) Assume that all our systems will be targeted by super hackers right now , and take appropriate measures to deal with that reality. If…

B) needs more time and A) provides exactly that.

Re: We must pace the frontier

#520

Post by bel8 »

qnleigh wrote: post #318
glub wrote: post #302

Dario, Sam, and Elon are all on the same page on this. So it's either they truly think AI is going to kill us all, or there's some other motives at play here. I don't think these people could possibly agree on the color of the sky, so what could the other possible motives be, based on what we know? OpenAI / Anthropic models have largely stopped advancing - that's not a good look when you're pre-IPO and Chinese models…

> OpenAI / Anthropic models have largely stopped advancing I'm shocked anyone could conclude this. This year it became common for people to entirely delegate coding to AI (I know many competent programmers/researchers who do this now). Progress in math has just been insane. An internal model at OAI just resolved one of the most celebrated open problems in mathematics. If anything, progress has accelerated.

I'm not. Yes we normalized 1m context window and models tend to hallucinate less.

But models have been somewhat stagnant since Opus 4.6/7.

And in some regards there were even regressions like Claudeisms that are load bearing.

Post reply on HN