Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We must pace the frontier

darioamodei.com

Post reply on HN
Print view View original

591–600 of 930 posts

Re: We must pace the frontier

#591

Post by haute_cuisine »

atombender wrote: post #557
academia_hack wrote: post #90

Dario's proposed approach is a classic example of capital attempting to control technological advancement and the means of production. For the first time in human history, any member of the working class can just about afford to have a team of expert scientist/physician/lawyer/engineers working directly for them. Super intelligence (the ability to have many smarter minds than your own reporting to you) has always bee…

If true AGI superintelligence is invented, I don't see how it will be made available to the masses, at least not intentionally. There will be no incentive for the likes of Anthropic and OpenAI to give such immense power to anyone else. Whoever has the superintelligence will be able to race ahead of everyone else. However, such technology may be leaked, or it may (as what happened with LLMs) be so simple to replicate…

ant/openai already keep their most advanced models behind closed doors and only share previous generation to the public

Re: We must pace the frontier

#592

Post by aswegs8 »

zozbot234 wrote: post #577
RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

The entire idea of RSI is completely speculative and unproven anyway - the whole underlying claim is that you could prompt a frontier model (at some unspecified level of smarts) to "think about ways to improve your own architecture" and this would then result in the model becoming infinitely smart ("superintelligent") via some sort of foolproof, unconstrained positive feedback. It's more of a science fictiony trope t…

Yeah but why shouldn't this be possible? We learned that we can already create artifical intelligence that surpasses human intelligence in some dimensions. There is no natural barrier here. The pace of this improvement would be debatable, but what speaks against the possibility of such accelerating self-improvement?

Re: We must pace the frontier

#593

Post by reducesuffering »

sidcool wrote: post #454

Altman and Musk have both shown support for this. Which itself is a worrying development.

It's worrying because they all agree, along with Turing Award winners like Geoffrey Hinton and Yoshua Bengio, that this is indeed an existential risk to humanity.

Re: We must pace the frontier

#594

Post by howunfortunate »

ah1508 wrote: post #489

Don't you think that AI hate will limit the general use and then revenues so it will slow down by itself while the niche (AlphaFold for instance) will remains ? Origins of AI hate: * "my boss wants me to use AI but he does not understand my job nor how AI works" * "AI will kill all of us" * "AI will destroy my job (or my colleague's job if I use AI better than him)". * I cannot pay my electricity bills because of AI…

I do not think hate alone usually slows things very much if they have economic utility (or people strongly believe they do)

It's the byproducts of hate (usually regulation, but occasionally things like boycotts or PR disasters) that do so. In the absence of those things, they just keep on truckin'

See: Bitcoin, Tesla

Re: We must pace the frontier

#595

Post by olelele »

snaking0776 wrote: post #527
Loquebantur wrote: post #470

Earlier quoted context omitted.

Your idea about the world being inhabited by mindless seekers of short-sighted "competitive advantages" is the fantasy. Getting people to agree isn't easy, that doesn't mean it wasn't worthwhile. There often even isn't any viable alternative to reaching an agreement in social settings. When your hyper-competitive mode of thinking leads to undesirable consequences, you have to change your mindset. Reality doesn't conf…

Yes I think this level of pessimism doesn’t actually hold up to the history of the last 100 years as it relates to nuclear weapons, cloning, wars etc. In fact I think the history actually points to the fact that believing such deals were impossible was a source of much of the strife. In many cases as soon as people started to actually speak to each other about what they had been fighting about they quickly realized t…

This is off topic now but yeah the world would have been a different place if someone had listened to the demands of Ho Chi Minh earlier.. and Hammarskjöld had not been crashed

Re: We must pace the frontier

#596

Post by zozbot234 »

aswegs8 wrote: post #592
zozbot234 wrote: post #577

Earlier quoted context omitted.

The entire idea of RSI is completely speculative and unproven anyway - the whole underlying claim is that you could prompt a frontier model (at some unspecified level of smarts) to "think about ways to improve your own architecture" and this would then result in the model becoming infinitely smart ("superintelligent") via some sort of foolproof, unconstrained positive feedback. It's more of a science fictiony trope t…

Yeah but why shouldn't this be possible? We learned that we can already create artifical intelligence that surpasses human intelligence in some dimensions. There is no natural barrier here. The pace of this improvement would be debatable, but what speaks against the possibility of such accelerating self-improvement?

> We learned that we can already create artifical intelligence that surpasses human intelligence in some dimensions.

Yes and this was very hard and required massive real-world resources. We didn't just get a sudden flash of insight by thinking real hard about how to make ourselves smarter. Yet that's always the story that underlies any claim of RSI. You can always phrase things generally enough to make any kind of AI-led improvement look like "RSI" no matter how short-term and tightly bounded, but that's just not helpful.

Re: We must pace the frontier

#597

Post by reducesuffering »

maxutility wrote: post #179

I’m disappointed in the level of groupthink reflexive cynicism I see from commenters any time prominent AI leaders talk about AI risks and the need for regulation or pacing. Yes, regulatory capture is a risk, but this is also a profoundly unusual, fast moving, and potentially extraordinarily dangerous technology. There are strict regulations around nuclear weapons, as well as around US financial, energy, and other in…

This thread is a testament to the travesty of how bad HN discourse has gotten. It's become an echo chamber of knee-jerk cynical sneering. All the heads of the labs (Sama, Dario, Demis Hassabis, Ilya, Musk) agree with the risks to humanity's existence. Most of the core employees agree. Yoshua Bengio, Geoffrey Hinton, Bernie Sanders agree. These issues have been discussed in more intelligent channels for a decade now, and still after all the incredible mathematics and coding progress that is improving incredibly fast every 6 months, this forum is fundamentally unserious. You will not find prescient views in the HN majority anymore, for many years now. The people intelligent, prescient, and forward thinking enough, skating to where the puck was going, to be involved in shaping the future of AGI are elsewhere and barely involved in this place from how badly it's gotten. The inventor of RLHF [0], and newest board member of OpenAI, wasn't on HN, they were on LessWrong.

[0] https://arxiv.org/abs/1706.03741

Re: We must pace the frontier

#598

Post by un_montagnard »

Anthropic is about to go public, but they figured they can't keep making improvement at the same pace they used to. If the frontier is paced, they can continue hyping their unreleased capabilities that coincidentally cannot be released due to things outside their control. Which buy them more time to try to improve the models.

Re: We must pace the frontier

#599

Post by WhenIsntItTrue »

haute_cuisine wrote: post #591
atombender wrote: post #557

Earlier quoted context omitted.

If true AGI superintelligence is invented, I don't see how it will be made available to the masses, at least not intentionally. There will be no incentive for the likes of Anthropic and OpenAI to give such immense power to anyone else. Whoever has the superintelligence will be able to race ahead of everyone else. However, such technology may be leaked, or it may (as what happened with LLMs) be so simple to replicate…

ant/openai already keep their most advanced models behind closed doors and only share previous generation to the public

[dead]

Re: We must pace the frontier

#600

Post by baq »

zozbot234 wrote: post #577
RGS1811 wrote: post #521

I don’t understand all the comments assuming that RSI is the real threat here. Dario is admitting that they failed to solve alignment. Without alignment, further improvements in capability turn LLMs into wanton felony generators. This call to pace the frontier is dressed up as altruism but it’s an admission that they cannot produce a marketable product better than what they have. Pacing the frontier means the US labs…

The entire idea of RSI is completely speculative and unproven anyway - the whole underlying claim is that you could prompt a frontier model (at some unspecified level of smarts) to "think about ways to improve your own architecture" and this would then result in the model becoming infinitely smart ("superintelligent") via some sort of foolproof, unconstrained positive feedback. It's more of a science fictiony trope t…

The idea that a few hundred apes with nothing but a bunch of rocks could one day land on the moon and come back to earth safely must’ve sounded ridiculous a hundred thousand years ago
Post reply on HN