Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

961–970 of 971 posts

Re: We must pace the frontier

#961

Post by tripzilch »

aswegs8 wrote: post #592
zozbot234 wrote: post #577

Earlier quoted context omitted.

The entire idea of RSI is completely speculative and unproven anyway - the whole underlying claim is that you could prompt a frontier model (at some unspecified level of smarts) to "think about ways to improve your own architecture" and this would then result in the model becoming infinitely smart ("superintelligent") via some sort of foolproof, unconstrained positive feedback. It's more of a science fictiony trope t…

Yeah but why shouldn't this be possible? We learned that we can already create artifical intelligence that surpasses human intelligence in some dimensions. There is no natural barrier here. The pace of this improvement would be debatable, but what speaks against the possibility of such accelerating self-improvement?

> We learned that we can already create artifical intelligence that surpasses human intelligence in some dimensions

yeah they're called calculators

Re: We must pace the frontier

#963

Post by aqme28 »

It's weird that we have two parallel discussions and they're not meeting in the obvious middle.

OpenAI is out there hacking HF, RubyGems, etc...

Dario is talking about restricting the frontier, pausing development.

The problem is that "restricting the frontier" benefits no one as much as the frontier labs. But there are other ways to force alignment-- make companies liable for these felonies their agents commit.

Make them liable, and they might actually devote the necessary resources to fixing these problems, rather than throwing their hands up and just saying "Trust nobody buy us", which obviously no one does.

Re: We must pace the frontier

#964

Post by a_ba »

Maybe instead of creating more rules we should trim down our legal systems and re-instate some basic accountability. I‘m pretty sure we have sent people to prison for running a botnet before and slapped material fines on torrenting operations.

Re: We must pace the frontier

#967

Post by tripzilch »

keeda wrote: post #842
hgoel wrote: post #554

Earlier quoted context omitted.

Why are we accepting the framing that the LLMs are felony generators, when the only incidences of LLM generated felonies involved misconfigured sandboxes and reckless waste of resources? The companies doing these things without following common sense security measures are the felony generators.

As TFA calls out, these agents were not asked to do any of these things and yet they did, at a bonkers scale, within just this handful of companies you mention. Whether they had leeway to is secondary to the fact that they did. Heck, they exploited zero day flaws which by definition means they went beyond common sense security measures. And now these agents are already being deployed all over the world at an ever inc…

> How much of the world do you think follows "common sense security measures"?

Well clearly all of them, cause so far it's only been this handful of companies running a felony-generator connected to a terminal and compute resources.

It would really help in these discussions if people wouldn't randomly jump between what actually happened and is happening, and things they envision/expect to happen at some point in the future ...

> these agents were not asked to do any of these things

no but they were clearly fine tuned to.

> at a bonkers scale

I mean let's not get hyperbolic

> they exploited zero day flaws which by definition means they went beyond common sense security measures

that's really not true. lots of common sense security measures protect against "zero day" flaws, it's called "defense in depth", and it was very much lacking

Re: We must pace the frontier

#968

Post by Gigachad »

cuuupid wrote: post #457

At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record, - no open weights - can’t use claude to research AI - train on everyone else’s IP and sell it back to them - 8 regulatory capture attempts and counting - so controlling they are the only US company blacklisted by the US government This is not effective altruism / rationalism gone wild, it’s just monopolisti…

> so controlling they are the only US company blacklisted by the US government

Their control here was refusing to make fully automated killing machines. They simply required someone to have to press the button.

Re: We must pace the frontier

#969

Post by andxor »

logicchains wrote: post #352
andxor wrote: post #180

Earlier quoted context omitted.

Really? Anthropic has the strongest models and it's in the best position to begin RSI and win the race. A pause would favor competitors.

Anthropic is far behind OpenAI now, that's why it's OpenAI that's solving Millennium Prize problems, and why Astra completely blows away Fable on benchmarks.

Fable is still better than Astra and Fable is not the best model Anthropic has.

Re: We must pace the frontier

#970

Post by tripzilch »

mapontosevenths wrote: post #855
adsharma wrote: post #785

Earlier quoted context omitted.

The real threat is that we uncritically adopt language such as alignment. Implicit in this is the idea that AI is a inscrutable matrix and going to remain that way and we'll need expert interpreters to make sense of it. We need to insist on building tech that's explainable by design.

> We need to insist on building tech that's explainable by design. You realize this means insisting on terrible tech that humans can understand right? It essentially caps human progress at some point about 4 years ago. If you are old and happy with the way things are this might sound like a good idea. It does not to me.

Ahh yes I remember the bad old days of 4 years ago when everyone decided human progress had enough, and we would have been stuck there forever if it hadn't been for LLMs ... we didn't know how good we had it
Post reply on HN