Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pacing the frontier

pacingthefrontier.com

Post reply on HN
Print view View original

81–90 of 216 posts

Re: Pacing the frontier

#81

Post by areoform »

esafak wrote: post #22
areoform wrote: post #10

The first computers were used for war. The Harvard Mark I was used by von Neumann to calculate the physics of implosion. And then right afterwards, ENIAC was used to calculate the feasibility of the H-bomb. They called it the "Los Alamos problem." The first computer network, SAGE (or, Semi-Automatic Ground Environment) brought together RADAR data from across the US with air-traffic control data and military flight da…

A lot of atomic scientists did regret their work.

Yes, and did you see them oppose computing? They promoted it and tried to use nuclear energy for peace.

Re: Pacing the frontier

#83

Post by biffles »

It's an admirable idea, but I don't see how you can slow this down in practice. We certainly know China will not, and they are already at the frontier.

RSI is too important and too tangible at this point for everyone to slow down. It's the equivalent of telling the world to stop developing nuclear weapons after Hiroshima. Just not going to happen, and in the meantime dramatically increases the risk that the power of AGI lands in the hands of the few (read: the political class) rather than empowers the people.

Re: Pacing the frontier

#84

Post by nostrebored »

rossant wrote: post #37
alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

This is serious. We're in uncharted territory. The geopolitical context is already complicated enough that we shouldn't be dismissive of such a rapid technological race. I'm glad AI leaders are taking it seriously. I'm surprised so many HN commenters aren't. On the other hand, I'm pessimistic about initiatives like this being effective on a global scale (though they could certainly work at the national level).

we're always in uncharted territory. that's why most HN commenters think that this is hyperbole.

Re: Pacing the frontier

#85

Post by aesthesia »

dmarcos wrote: post #45
red_green_yell wrote: post #41

Earlier quoted context omitted.

See printing press and internet!

And video games, democracy, novels, TV, cars, photography, cryptography, the Beatles… pick your favorite and read commentary / opinions of the time. Many thought that came with unique risks never seen before and must be stopped!

How many of those concerns came from the people building the technologies? John Carmack wasn't up in arms about how Doom was going to destroy society. Charles Dickens didn't think novels rotted the brain. Johannes Gutenberg wasn't concerned about the effects of a free press.

Re: Pacing the frontier

#86

Post by marcus_holmes »

bpascal wrote: post #49
red_green_yell wrote: post #32

Earlier quoted context omitted.

Are they lazy dismissals? Or a simple observation that LLM scaling is hitting diminishing returns way way short of AGI. If the claim is (1) AGI coming and (2) we should so something about AGI, wouldn't it be smart to confirm (1) before worrying about (2)? Especially if certain companies have trillions of dollars of valuation that depend on society as a whole believing (1)?

i think part of the problem, though, is that we've already hit many of the warning signs that AGI is coming, and people have kept moving the goalposts. if you were to tell someone from ten years that there's an ai that has passed the turing test and has made novel mathematical discoveries, i think many would say AGI is already here.

You're right, in that we would have said that.

I wouldn't frame it as moving the goalposts in a bad way, though. We understand more now than we did then, so we have a better idea of what the actual goal is, and moving the goalposts to frame that goal is more rational than insisting the goalposts stay where they were ten years ago even though the entire pitch has changed since then. I'm straining the analogy, obviously.

Re: Pacing the frontier

#87

Post by yks »

> We request that the U.S. government ...

Too bad these researcher's organizations spent their efforts supporting the dumbest, ficklest possible version of the U.S. government to guide us through the gravity of this moment.

Re: Pacing the frontier

#88

Post by versteegen »

areoform wrote: post #77
philipkglass wrote: post #46

Earlier quoted context omitted.

I know a lot more about chemistry and physics than I know about biology, so biological risk is still at least a "maybe" reason to be worried about model safety for me. I don't worry about other weapons of mass destruction developed with LLM assistance. Nuclear weapons are already controlled much more with political pressure and the physical difficulty of e.g. enriching uranium than they are controlled by secret-keepi…

This experiment has been run, and even the biological stuff is far fetched. I've written about this before, but the issue is that these people have worked with computers their entire lives, and they keep projecting outwards from there. The 20th century was dominated by mad scientists (mostly Teller) and generals (bombs away LeMay), but one thing that I respect about them is that they didn't just sit and guess about p…

Thank you for taking this seriously enough to write this, and anyone else likewise.

But this is attacking a strawman, amateur bioterrorists. AI is a force multiplier in the hands of an expert. If it took a team before, maybe a single malicious actor can accomplish it now that AI can fill in the parts they aren't well-versed in. And that dramatically increases the chance of it happening.

Being at risk of killing yourself also just makes success X% less likely, but if X < 90 that doesn't mitigate much.

Re: Pacing the frontier

#89

Post by OutOfHere »

ajyoon wrote: post #65
OutOfHere wrote: post #8

Pacing is a really bad idea because China will not slow it down for you.

The entire point of the statement is to start international coordination.

It will never happen because there is money to be made by not coordinating, i.e. by leaping ahead. It also doesn't make any sense to hold AI back.

Re: Pacing the frontier

#90

Post by OutOfHere »

levocardia wrote: post #68
vessenes wrote: post #61

This. Will. Never. Work. This research game is a zero-trust iterated game. Any defection carries the benefit of possibly being permanently valuable. There will always be defections. OpenAI defected WHILE THEY WERE pushing to slow. The cost to the US of deciding to slow is so massively harmful that it is wrong to even put this idea out in the public zeitgeist; it may somehow become a thing that US legislators believe…

Why not? Slowing down / halting biological weapons research mostly worked. Yes there have probably been modest sized defections here and there, but the pace of bioweapon development is a crawl compared with (a) what is possible with science already, and also with (b) the pace of bioweapons development from 1910 to 1970.

Bioweapons don't make money. AI does. The two are therefore not comparable.
Post reply on HN