Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pacing the frontier

pacingthefrontier.com

Post reply on HN
Print view View original

41–50 of 216 posts

Re: Pacing the frontier

#41

Post by red_green_yell »

dmarcos wrote: post #36
alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

Every single technology in history has had people claiming potential existential risks that didn’t materialize. Burden of proof is on people making the risk claims. Those have to be specific and unique to that technology to grant special actions.

See printing press and internet!

Re: Pacing the frontier

#42

Post by [deleted] »

alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

[deleted]

Re: Pacing the frontier

#43

Post by MeetingsBrowser »

brap wrote: post #38

Has there ever been a case in human history when technology just stopped advancing, simultaneously world wide, only because a bunch of people wanted it to? I think these people are either incredibly naive, or, more likely, not acting in good faith.

DNA splicing, human cloning, super-sonic passenger flight, human gene editing (CRISPR), nuclear energy, biological weapons, and probably more.

Re: Pacing the frontier

#44

Post by ece »

What these people want is same thing people in every other field want too, which is being able to use AI as a tool and nothing more. Off course, some of the people that signed this might not recognize this, and the people who didn't sign it are actively working against such an equilibrium.

Re: Pacing the frontier

#45

Post by dmarcos »

red_green_yell wrote: post #41
dmarcos wrote: post #36

Earlier quoted context omitted.

Every single technology in history has had people claiming potential existential risks that didn’t materialize. Burden of proof is on people making the risk claims. Those have to be specific and unique to that technology to grant special actions.

See printing press and internet!

And video games, democracy, novels, TV, cars, photography, cryptography, the Beatles… pick your favorite and read commentary / opinions of the time. Many thought that came with unique risks never seen before and must be stopped!

Re: Pacing the frontier

#46

Post by philipkglass »

alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

I know a lot more about chemistry and physics than I know about biology, so biological risk is still at least a "maybe" reason to be worried about model safety for me. I don't worry about other weapons of mass destruction developed with LLM assistance. Nuclear weapons are already controlled much more with political pressure and the physical difficulty of e.g. enriching uranium than they are controlled by secret-keeping.

For all I know (I don't know a lot about biology), it's possible to develop a virus that spreads as easily as measles, but has the lethality and long induction period of HIV. If that's true, and if LLMs would make it easy to develop such a virus, I see how no-guardrails LLMs could be truly disastrous.

The other justifications for restricting model training don't seem nearly as compelling. Worse, restrictions on model training have an infectious quality if they mean something more than "leading AI labs will voluntarily slow down together." (Like if the signatories mean that better models and software for building them should be regulated.) It's too dangerous to publish stronger models. So it's also too dangerous to publish training software. So it's also too dangerous to publish tools that allow you to build training software. (Claude's Fable won't help you write generic model training pipelines for this very reason.)

If you keep following the implications of this safety argument, it's as broad an assault on the distribution of software and computing as has ever been proposed.

Re: Pacing the frontier

#47

Post by ubertaco »

alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

We're commenting on a site run by a Silicon Valley tech start-up accelerator. "Move fast and break things" is the accelerationist way, even if the "things" they break are a stable, functioning social order. As long as you IPO and cash out along the way, you can wreck everyone else's lives just fine, in the minds of this sort.

Re: Pacing the frontier

#48

Post by reasonableklout »

brap wrote: post #38

Has there ever been a case in human history when technology just stopped advancing, simultaneously world wide, only because a bunch of people wanted it to? I think these people are either incredibly naive, or, more likely, not acting in good faith.

https://en.wikipedia.org/wiki/Asilomar_Conference_on_Recombi...

> The effects of these guidelines are still being felt through the biotechnology industry and the participation of the general public in scientific discourse. Due to potential safety hazards, scientists worldwide had halted experiments using recombinant DNA technology, which entailed combining DNAs from different organisms. After the establishment of the guidelines during the conference, scientists continued with their research, which increased fundamental knowledge about biology and the public's interest in biomedical research.

Re: Pacing the frontier

#49

Post by bpascal »

red_green_yell wrote: post #32
alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

Are they lazy dismissals? Or a simple observation that LLM scaling is hitting diminishing returns way way short of AGI. If the claim is (1) AGI coming and (2) we should so something about AGI, wouldn't it be smart to confirm (1) before worrying about (2)? Especially if certain companies have trillions of dollars of valuation that depend on society as a whole believing (1)?

i think part of the problem, though, is that we've already hit many of the warning signs that AGI is coming, and people have kept moving the goalposts. if you were to tell someone from ten years that there's an ai that has passed the turing test and has made novel mathematical discoveries, i think many would say AGI is already here.

Re: Pacing the frontier

#50

Post by dzink »

The historic analog most befitting AI are weapons. It provides an advantage to companies and governments in creating weapons, in surveilling people and thus gaining leverage over individuals, in controlling what individuals can do or cannot stop doing. It thus follows the laws of weapons control - the more restrictions are threatened to be imposed, the more even the smallest actors will fight to get more of it. They may not understand or know what they are getting, but they will want more of it because their potential rivals have it. Governments are the ultimate customer here - they have the money, the energy, and the appetite to consume AI infinitely in battlement against opponents - foreign and domestic. The only approximate solution is to prevent any restrictions on the use of it - a-la-2nd amendment (the upside of that is it advances the AI for future use and is thus in the interest of governments as well). Modern large models seem to be compressible and thus we may see more and more tiny sub-specialized varieties. But the push to slow the frontier in weaponry always leads to the reverse.
Post reply on HN