Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

991–1000 of 1001 posts

Re: We must pace the frontier

#991

Post by boshalfoshal »

eknkc wrote: post #868
boshalfoshal wrote: post #857

Earlier quoted context omitted.

I think you fundamentally misunderstand what kind of company Anthropic is. I'm simply taking the path of least resistance when understanding what people at Anthropic have to say, and basically every public source points to Anthropic talking the talk as well as walking the walk with regards to AI safety and their ideas on AI governance. Re the point on returning shareholder value, Anthropic is a PBC: > Public Benefit…

Oh I am terribly sorry, did not know Anthropic was formed as a a “Good Company”. Which changes everything ever. Snark aside, maybe I’m too pessimistic and you are correct on this. But.. doesn’t that mean we are completely fucked? If they don’t have any ulterior motive and they truly are panicking.. We should start packing.

yes that is literally why they are sounding the alarm - because people think AI is a party trick and they aren't taking it seriously enough.

Re: We must pace the frontier

#992

Post by boshalfoshal »

qlte wrote: post #980
boshalfoshal wrote: post #857

Earlier quoted context omitted.

I think you fundamentally misunderstand what kind of company Anthropic is. I'm simply taking the path of least resistance when understanding what people at Anthropic have to say, and basically every public source points to Anthropic talking the talk as well as walking the walk with regards to AI safety and their ideas on AI governance. Re the point on returning shareholder value, Anthropic is a PBC: > Public Benefit…

The decision to found Anthropic to compete with OpenAI's then concentration of SOTA LLM AI researchers, and then achieving/surpassing parity with OpenAI is pretty much directly responsible for creating the current capabilities arms race. Google has shown they are not motivated at all to push the bigger and bigger SOTA frontier with Mythos-type super models. Even OpenAI isn't really even trying to surpass Anthropic at…

If you've read enough lesswrong, this won't be new to you. The strategy always was "become the most capable model lab so that you have leverage to control AI development."

Re: We must pace the frontier

#993

Post by throw10920 »

kvuj wrote: post #989
throw10920 wrote: post #987

Earlier quoted context omitted.

> - no open weights That's a good thing, not a bad thing. Nobody is entitled to give you something that they spent (tens of?) billions of dollars to develop for free. > - can’t use claude to research AI No problems there either. > - train on everyone else’s IP and sell it back to them Which the courts have yet to actually make a ruling on. It seems like it's perfectly legal. > - 8 regulatory capture attempts and coun…

> That's a good thing, not a bad thing. Nobody is entitled to give you something that they spent (tens of?) billions of dollars to develop for free. But the Chinese are doing just that? > Which the courts have yet to actually make a ruling on. It seems like it's perfectly legal. Legal != moral

> But the Chinese are doing just that?

Completely and totally irrelevant. Has nothing to do with the fact that you are not obligated to others' work.

> Legal != moral

Sure, you're right.

I guess the bigger issue is that the GP's post calls out Anthropic for following along in legal (but what we both believe to be immoral) mass copyright infringement that OpenAI started, while absolving them of the exact same thing. It's a shill account.

Re: We must pace the frontier

#994

Post by emodendroket »

RGS1811 wrote: post #493

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

You don't see this as an attempt to create a new regulatory moat then?

Re: We must pace the frontier

#995

Post by rphv »

glub wrote: post #287

Dario, Sam, and Elon are all on the same page on this. So it's either they truly think AI is going to kill us all, or there's some other motives at play here. I don't think these people could possibly agree on the color of the sky, so what could the other possible motives be, based on what we know? OpenAI / Anthropic models have largely stopped advancing - that's not a good look when you're pre-IPO and Chinese models…

> So it's either they truly think AI is going to kill us all

Yes. Basically everyone here is dismissing this possibility. We shouldn’t.

Re: We must pace the frontier

#996

Post by ta20240528 »

glub wrote: post #287

Dario, Sam, and Elon are all on the same page on this. So it's either they truly think AI is going to kill us all, or there's some other motives at play here. I don't think these people could possibly agree on the color of the sky, so what could the other possible motives be, based on what we know? OpenAI / Anthropic models have largely stopped advancing - that's not a good look when you're pre-IPO and Chinese models…

"So it's either they truly think AI is going to kill us all"

Exactly how does a CPU kill 8 odd billion people? Even if they got hold of the weapons to wipe out a few billion, surely the remain billions would just turn off the data centres?

Re: We must pace the frontier

#997

Post by maxnevermind »

Hmm, what a coincidence both OpenAI and Anthropic suggest to slow down. I think the signal it sends in plain English sounds something like: We burn too much money atm and shit is about to hit the fan with all that compute capacity starting to rapidly come online 2027, we are not seeing enough demand to justify all of those data centers we already agreed to pay for, let's do a damage control and lower expectation of the plebs.

Re: We must pace the frontier

#998

Post by aryehof »

RGS1811 wrote: post #493

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

> Without alignment, further improvements in capability turn LLMs into wanton felony generators

Isn’t “full” alignment and guardrails a task that can never be generally achieved? One will always discover and realize the need for new guidelines and guardrails? What about vague, incomplete, inconsistent nuances and generalizations - that make this a losing battle?

I suggest ultimate safeguards need to be outside the models?

Re: We must pace the frontier

#999

Post by Valakas_ »

atombender wrote: post #525
academia_hack wrote: post #90

Dario's proposed approach is a classic example of capital attempting to control technological advancement and the means of production. For the first time in human history, any member of the working class can just about afford to have a team of expert scientist/physician/lawyer/engineers working directly for them. Super intelligence (the ability to have many smarter minds than your own reporting to you) has always bee…

If true AGI superintelligence is invented, I don't see how it will be made available to the masses, at least not intentionally. There will be no incentive for the likes of Anthropic and OpenAI to give such immense power to anyone else. Whoever has the superintelligence will be able to race ahead of everyone else. However, such technology may be leaked, or it may (as what happened with LLMs) be so simple to replicate…

If true AGI/superintelligence is invented, humans will not remain in control and "give access". People don't seem to grasp what it actually means to create something more intelligence than us. A monkey can't give human access to other monkeys, because no matter what a monkey does, a human will always outsmart him and be the one in control. If we create AGI, there is not such thing as "giving access" anymore.

Re: We must pace the frontier

#1000

Post by rphv »

ta20240528 wrote: post #996
glub wrote: post #287

Dario, Sam, and Elon are all on the same page on this. So it's either they truly think AI is going to kill us all, or there's some other motives at play here. I don't think these people could possibly agree on the color of the sky, so what could the other possible motives be, based on what we know? OpenAI / Anthropic models have largely stopped advancing - that's not a good look when you're pre-IPO and Chinese models…

"So it's either they truly think AI is going to kill us all" Exactly how does a CPU kill 8 odd billion people? Even if they got hold of the weapons to wipe out a few billion, surely the remain billions would just turn off the data centres?

Bioweapons, trivially. All it would need to do is convince one person who works in a lab, possibly under the guise of benign intent.

Or... something so far outside of the box that we'd have about as much understanding of it as a baboon does a gun.

Post reply on HN