Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pacing the frontier

pacingthefrontier.com

Post reply on HN
Print view View original

141–150 of 216 posts

Re: Pacing the frontier

#141

Post by 0xDEAFBEAD »

NitpickLawyer wrote: post #118
alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

> We should want the AI Labs to make binding agreements to proceed safely and slower That's a red herring, IMO. Once the cat is out of the bag, there's no "slowing down", only "slowing up". What's even more egregious is that the people behind the "slowing down" camp are also the people who've had 10s of billions invested in their companies. It smells of "we got ours, now you can't follow". It reeks of regulatory capt…

>What's even more egregious is that the people behind the "slowing down" camp are also the people who've had 10s of billions invested in their companies. It smells of "we got ours, now you can't follow". It reeks of regulatory capture, which is a proven net downside for everyone else.

This sort of cynicism feels un-falsifiable.

If companies advocate against regulation, the cynics will say: "These corporations want capitalism to run amok with no regulation to interfere with their profits. They need to go full steam ahead in order to show profits for their investors."

If companies advocate in favor of regulation, the cynics will say: "Regulatory capture."

If you have legitimate concerns, how about a constructive discussion regarding how those concerns could be addressed, in light of the "risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems" which over 1000 frontier AI company employees expressed worry about?

I don't think it's justified to jump immediately to cynicism, if you haven't even made a token effort at good-faith engagement.

>The moment they started adding "nucular" to the acronyms of "safety alignment" you could tell this is just kabuki at best. There's absolutely nothing that a chatbot will tell you that will get you closer to anything nuclear. It's a joke.

What are you referring to? Are you referring to this quote from the OP?

"The world is locked in a deadly race towards an intelligence explosion, where AI’s ability to create better AIs reaches a critical point, just like a runaway nuclear chain reaction. Going slower would give us much-needed time to make it go well, but no individual actor is willing to stop unilaterally. To survive, we must coordinate to slow down the race."

Your response seems to be a total non sequitur, or failure to understand Leo's point. Exponential growth can easily get out of control, as we saw early in COVID. The point about nuclear is mainly an analogy.

Re: Pacing the frontier

#142

Post by throwaway13337 »

Implied in this is a concentration of power required to enforce.

No one is sold on a dystopian hellscape of fear, surveillance, and control. Each one of these would be required for a select group to effectively enforce AI safety.

Inflicting fear on society has never lead anywhere good. Please stop.

New technologies have always been scary. But humans have adapted. I believe in humanity. Do you?

Re: Pacing the frontier

#143

Post by 0xDEAFBEAD »

txrx0000 wrote: post #130
NitpickLawyer wrote: post #118

Earlier quoted context omitted.

> We should want the AI Labs to make binding agreements to proceed safely and slower That's a red herring, IMO. Once the cat is out of the bag, there's no "slowing down", only "slowing up". What's even more egregious is that the people behind the "slowing down" camp are also the people who've had 10s of billions invested in their companies. It smells of "we got ours, now you can't follow". It reeks of regulatory capt…

We have to stop comparing nuclear weapons with general intelligence. They are nothing alike. Nukes are weapons of mass destruction that can only destroy or deter. Intelligence is dual-use/multi-use, with no restrictions on strategy. If we look at all organisms on Earth, species that are more intelligent are also more prosperous. This is true from bacteria, to insects, to animals, to humanity. Intelligence itself is n…

>Nukes are weapons of mass destruction that can only destroy. Intelligence is dual-use.

Perhaps a better analogy, instead of "nukes", would be "nuclear technology".

Just like intelligence, nuclear technology is dual-use: Capable of both energy generation which fuels prosperity, and also mass destruction.

Just like intelligence, nuclear technology demands a healthy level of caution and vigilance.

>If we look at all organisms on Earth, species that are more intelligent are also more prosperous. This is true from bacteria, to insects, to animals, to humanity. Intelligence itself is not harmful.

Human intelligence has been quite harmful to the species we've driven extinct! AI intelligence could be similar for our species.

>There will be new adversarial games when everyone's smarter (bio/cyber offense/defense), but those games are always symmetrical in the long run

We've had multiple near misses with nuclear technology, such as the Cuban Missile Crisis. I think you're presuming an inevitability which just doesn't exist.

Same way a person living in the year 1500 would have no hope of accurately predicting life in the year 2000, a person living today has no hope of accurately predicting where technological advances will take us, in the limit. This inherent uncertainty should, again, fuel caution and vigilance.

Human technology is already so far beyond that of other species to make us a huge outlier. Even if intelligent species are more prosperous as a general evolutionary trend, it would be irresponsible to extrapolate that trend light years from the domain where most of your observations are.

Almost all the species which have lived on Earth have gone extinct. It could happen to us too.

Re: Pacing the frontier

#144

Post by frabcus »

intorio wrote: post #69
ajyoon wrote: post #65

Earlier quoted context omitted.

The entire point of the statement is to start international coordination.

The problem is how do you enforce coordination? This isn't like nuclear testing that can be detected around the world on seismographs or from satellites in space. AI research happens in benign looking buildings that could be doing a hundred different non-AI workloads, much of it things that industry or governments would not want 3rd parties prying into. There is too much to potentially gain to trust coordination.

See detailed supplement to Plan A which talks entirely about this:

https://ai-2040.com/supplements/verification-plan

Re: Pacing the frontier

#145

Post by frabcus »

intorio wrote: post #115
ajyoon wrote: post #109

Earlier quoted context omitted.

There is a lot of research out there with answers to your questions. GPU supply chains are extremely constrained, making it a surprisingly tractable problem at least for now.

How do you tell the difference between inference and training workloads from outside? The same GPUs can be used for both and the pacing suggested here relates to training and not inference as far as I can tell.

Random sampling of compute activity.

https://ai-2040.com/supplements/verification-plan#appendix-p...

Re: Pacing the frontier

#146

Post by throwaway13337 »

dzink wrote: post #50

The historic analog most befitting AI are weapons. It provides an advantage to companies and governments in creating weapons, in surveilling people and thus gaining leverage over individuals, in controlling what individuals can do or cannot stop doing. It thus follows the laws of weapons control - the more restrictions are threatened to be imposed, the more even the smallest actors will fight to get more of it. They…

I think we see things similarly about the negative potential but please be careful with the framing.

AI is a printing press or a steam engine. It's not a weapon.

The framing is important. Much like 'distillation attack'. These framings are an attack on the narrative.

Printing presses at the time were seen as extremely dangerous too. For many of the same reasons cited about AI.

In both cases, the technologies were used to destablize existing power structures. And, after some hardship, make a better world for most people.

Even calling these things 'dual use' is a weird framing. Books are not 'dual use technologies' unless you want to sell fear.

Re: Pacing the frontier

#147

Post by 0xDEAFBEAD »

soundworlds wrote: post #126
alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

It really comes down to this: Sam and Dario are leading companies that who made the reckless decision to train on everyone's IP without proper process or consent. It has been proven they trained on pirated content. There is no question about this. Then, as soon as there are open models who threaten their businesses, they cry "distillation and IP theft". It's the last 3 years of pure hypocrisy and fear-baiting for inv…

You're missing the point of the OP:

"there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems... each company—and country—is under intense competitive pressure not to unilaterally slow that acceleration"

>To be honest, I think people would now prefer the risk of misused open models, when the alternative is giving these guys more power.

Then advocate a muscular regulatory regime which reduces the amount of power they have. Tell them: "You want regulation? OK. How about everyone whose IP you trained on gets some sort of veto option on your models? That should reduce the racing risk that you're talking about."

Re: Pacing the frontier

#148

Post by 0xDEAFBEAD »

dullcrisp wrote: post #73
Davidzheng wrote: post #70

Earlier quoted context omitted.

Not just engineers - I see people attributing it to strong priors that nothing happens but in my opinion it's more some form of psychological defense mechanism against understanding there will be an intelligence vastly superior to humans which is a deeply uncomfortable thought (to many).

“Nothing ever happens” is like “you can’t beat the market.” It’s basically tautologically the strongest predictive model that we can have. A corollary is that something may happen, but it won’t be what you think.

Interesting fact. Bryan Caplan is a economist who became famous for making bets on the future. He would take the "nothing ever happens" side of the bet, and he would almost always win. In 2023 he took the "nothing ever happens" side of an AI bet. Just 6 months later, he wrote:

"When the answers change, I change my mind [...] Base rates have clearly failed me. I’m not conceding the bet, because I still think there’s a 10-15% chance I win via luck... make no mistake, this software truly is the exception that proves the rule."

https://www.betonit.ai/p/gpt-retakes-my-midterm-and-gets-an

At this point, straightforward trend extrapolation suggests that AI continues to get better and better. Arguably, continued rapid AI progress is the "nothing ever happens" bet at this point. If AI progress stops, or slows dramatically, something will have happened.

Re: Pacing the frontier

#149

Post by [deleted] »

peyton wrote: post #74
Davidzheng wrote: post #70

Earlier quoted context omitted.

Not just engineers - I see people attributing it to strong priors that nothing happens but in my opinion it's more some form of psychological defense mechanism against understanding there will be an intelligence vastly superior to humans which is a deeply uncomfortable thought (to many).

Why should I trust these randos instead of Claude? I know Claude. I would hope Claude is smarter than whoever wants to make rules about my use of Claude. Replace Claude with Kimi, etc. if needed. I don’t find a petition from people I don’t know who don’t care about me more compelling than my own direct experience with these models.

[deleted]

Re: Pacing the frontier

#150

Post by 0xDEAFBEAD »

txrx0000 wrote: post #104
atleastoptimal wrote: post #51

Earlier quoted context omitted.

So many otherwise smart engineers, overrepresented on HN, have this weird mental block where they fail to take AI progress seriously. I think it's a fear that their status-defining skill could be commodified, so they adopt a mental framework that aims to dismantle the seeming legitimacy of any genuine leap in AI progress as some sort of mirage

It's ironic that the top reply to a comment about quality of discourse on this site is an ad hominem. Most people here take AI progress very seriously judging by the number of posts on the front page about AI at any given time. And I think some engineers even explicitly want to commodify their "status-defining skill": https://nitter.net/__tinygrad__

>It's ironic that the top reply to a comment about quality of discourse on this site is an ad hominem.

You might find the Ad Hominem Fallacy Fallacy page interesting:

"Argumentum ad hominem is the logical fallacy of attempting to undermine a speaker's argument by attacking the speaker instead of addressing the argument. The mere presence of a personal attack does not indicate ad hominem: the attack must be used for the purpose of undermining the argument, or otherwise the logical fallacy isn't there. It is not a logical fallacy to attack someone; the fallacy comes from assuming that a personal attack is also necessarily an attack on that person's arguments."

https://laurencetennant.com/bonds/adhominem.html

So for example, if I said: "The arguments for constructing an AI pause button are bad because they are being made by AI company employees", this would be a fallacious argument.

If I said: "HN users have this weird mental block where they fail to take AI progress seriously", that's not necessarily a fallacious argument, unless it's for the purpose of undermining an argument the HN user made.

Post reply on HN