Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pacing the frontier

pacingthefrontier.com

Post reply on HN
Print view View original

211–216 of 216 posts

Re: Pacing the frontier

#211

Post by 0xDEAFBEAD »

red_green_yell wrote: post #207
0xDEAFBEAD wrote: post #153

Earlier quoted context omitted.

>frontier labs that have trillions of dollars of vested interests in this message. OP suggests the opposite: "each company—and country—is under intense competitive pressure not to unilaterally slow that acceleration."

OAI and Anthropic specifically are under intense pressure to compete. One form of competition could be to innovate. Another form would be regulatory capture. The former seems unlikely given how quickly open labs are catching up. The latter is much easier and cheaper. If you can't beat them, ban them.

"We request that the U.S. government support an international effort" (emphasis mine)

China is unlikely to sign on to an "international effort" which unfairly privileges US models.

Re: Pacing the frontier

#212

Post by 0xDEAFBEAD »

txrx0000 wrote: post #205
holmesworcester wrote: post #198

Earlier quoted context omitted.

> Release the base models and let the world do whatever with it. That will automatically produce the aligned scenario. For sufficiently smart base models, the aligned scenario this automatically produces is aligned with who? Not with the humans, with the base models. It seems that some people find this hard to imagine, but it's just a direct consequence of what intelligence is, and what any known agentic training pro…

Bio-digital integration will tighten in the meantime. I suspect neural interfaces will have blurred the line between my model and my brain by the time the model is wildly superintelligent, so it would become me (or I would become it? no difference at that point). Though the pitfall here is ownership of hardware - everyone will have to design and fab their own hardware if we do not want to become a hivemind. We will h…

Yeah I guess I value the human race too much to entrust it to hopes and prayers of the form: "I suspect neural interfaces will have blurred the line between my model and my brain"

You wouldn't get on an airplane if the pilot told you: "I suspect we will be able to safely land the plane"

With more people on the aircraft, your safety expectations should get stronger, not weaker.

Re: Pacing the frontier

#213

Post by HarHarVeryFunny »

mofeien wrote: post #17
fidotron wrote: post #15

Anyone serious about this should participate in pacing the frontier by quitting their job so as to not potentially contribute to the problematic progress. Be the change you want to see! Curious how that isn't what happens.

That's an application of the "Tragedy of the commons": One person stopping to push the frontier just means that this person will lose their job, the frontier will be pushed by the others. The solution is to push for a binding rule instead that will make everyone slow down in the racing towards artificial superintelligence.

That may be a practical argument, but it's certainly not a moral one - it's like justifying selling drugs on the basis that if you don't do it somebody else will.

If these frontier lab employees are actually sincere about wanting to do something about the risk of AI advancing faster than our ability to adapt to it, then what they should really be doing is lobbying their own employers to do something about it. Lead by example, and do something you have control over.

Lobbying the government to stop yourself from doing something bad(!!) can't be taken as a serious position - it can only be taken as a plea to get the competition (China - open weights) shut down. It makes no sense to claim that China is only keeping up by copying you, yet China, not yourself, is the problem.

Re: Pacing the frontier

#214

Post by QuantumGood »

alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

  > should want [others] to make binding agreements
As Munger emphasized, others act according to incentives. First, clarify those incentives, then incentivize them if you can.

Re: Pacing the frontier

#215

Post by Davidzheng »

dullcrisp wrote: post #73
Davidzheng wrote: post #70

Earlier quoted context omitted.

Not just engineers - I see people attributing it to strong priors that nothing happens but in my opinion it's more some form of psychological defense mechanism against understanding there will be an intelligence vastly superior to humans which is a deeply uncomfortable thought (to many).

“Nothing ever happens” is like “you can’t beat the market.” It’s basically tautologically the strongest predictive model that we can have. A corollary is that something may happen, but it won’t be what you think.

This is just objectively false - in fact your answer basically contradicts itself because the market reacts to indicators, if "nothing ever happens" was the best strategy, then the market (which is also best) must agree with it.

Re: Pacing the frontier

#216

Post by testerteert000a »

areoform wrote: post #80
testerteert000a wrote: post #60

Earlier quoted context omitted.

> ...I see how no-guardrails LLMs could be truly disastrous... by all accounts this is what's what because of how... > You can do CRISPR in your kitchen. .. and because of the same dynamic we see occurring in cybersec. If you're good enough at cybersec, but not good-good at specific skillsets like offensive, then LLMs and a comp sci degree makes you quite capable in the ways that count. This is a fact, full stop. If…

If you're good enough at cybersec, but not good-good at specific skillsets like offensive, then LLMs and a comp sci degree makes you quite capable in the ways that count. This is a fact, full stop. Wet work in a lab isn't the same as writing code on a computer. If you screw up with a lethal pathogen in a lab, you die. If you screw up with code on a computer, you go get lunch. They're not the same.

If you develop LLM-driven exploits that can work through a water treatment/ purification system, looks an awful lot the same.
Post reply on HN