Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pacing the frontier

pacingthefrontier.com

Post reply on HN
Print view View original

121–130 of 216 posts

Re: Pacing the frontier

#121

Post by Smaug123 »

fidotron wrote: post #53
philipkglass wrote: post #46

Earlier quoted context omitted.

I know a lot more about chemistry and physics than I know about biology, so biological risk is still at least a "maybe" reason to be worried about model safety for me. I don't worry about other weapons of mass destruction developed with LLM assistance. Nuclear weapons are already controlled much more with political pressure and the physical difficulty of e.g. enriching uranium than they are controlled by secret-keepi…

> For all I know (I don't know a lot about biology), it's possible to develop a virus that spreads as easily as measles, but has the lethality and long induction period of HIV. If that's true, and if LLMs would make it easy to develop such a virus, I see how no-guardrails LLMs could be truly disastrous. This is the specific example in the latest doomerist wave I have so much trouble with: by what mechanism is an LLM…

> cannot do spatial reasoning

Opus 4.8 was the first model that could plausibly roleplay a BJJ roll without making a physically implausible move within five conversational turns. Fable 5 was the first model that beat me at the task, where I was the first party to write down an implausible move.

Re: Pacing the frontier

#122

Post by overfeed »

alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

> It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest.

It's not a shallow dismissal if you have been following their - let's charitably call it "evolving" - public posture on AI legislation.

It wasn't long ago the same people/companies successfully lobbied the Trump admin to quash state laws that threatened to "Pace" AI, and set ground-rules for safety. Back then, open source models were very far behind the frontier, and their spiel then was it was imperative not to impinge on AI progress with red-tape, lest America's adversaries catch up.

I challenge you to explain what triggered the about-face. The simplest explanation is that open source/Chinese models have caught up with the frontier, and the change in opinion is tactical (and obviously self-interested), rather than a deeply held belief.

Re: Pacing the frontier

#123

Post by SturgeonsLaw »

sm-silversight wrote: post #101
OkayPhysicist wrote: post #56

Earlier quoted context omitted.

You can do CRISPR in your kitchen. It's been possible practically as long as CRISPR has been a thing. It's not particularly difficult, to the point that my biggest counterpoint to the whole AI doomerism argument on the subject is that nobodies done it yet. You don't need some genius AI to design a bioweapon, you need to do a bit of light reading on bioengineering. The fact that some mad scientist hasn't ended civiliz…

Can you point me at this light reading? I don't wanna do CRISPR in my kitchen, but itd be cool to know more about it. I thought I needed to be basically pre-med to grasp it.

Here's a step by step guide on how to gene edit rice

https://bio-protocol.org/en/bpdetail?id=1225&type=0

Mostly it's just heating and cooling tiny volumes of liquids

Re: Pacing the frontier

#124

Post by overfeed »

roywiggins wrote: post #66
red_green_yell wrote: post #32

Earlier quoted context omitted.

Are they lazy dismissals? Or a simple observation that LLM scaling is hitting diminishing returns way way short of AGI. If the claim is (1) AGI coming and (2) we should so something about AGI, wouldn't it be smart to confirm (1) before worrying about (2)? Especially if certain companies have trillions of dollars of valuation that depend on society as a whole believing (1)?

What if "way short of AGI" is still dangerous in ways we can't easily handle?

When did the pacers have this mass-revelation about dangers of AI? The timing is curious.

Which takes priority: public safety or the free-market + national security? The AI labs hype in now biting them in the ass: why would China and the rest of the world deny their businesses a technology that "is bigger than the invention of electricity" or whatever hyperbole the snake-oil salesmen came up with when they were not breathlessly declaring that the first side to get AGI wins forever.

Even if the pacers are right, it's like greenhouse emissions all over again; each country wants everyone else to cut emissions, but no one wants the economic hit of doing so.

Re: Pacing the frontier

#125

Post by versteegen »

red_green_yell wrote: post #114
versteegen wrote: post #88

Earlier quoted context omitted.

Thank you for taking this seriously enough to write this, and anyone else likewise. But this is attacking a strawman, amateur bioterrorists. AI is a force multiplier in the hands of an expert. If it took a team before, maybe a single malicious actor can accomplish it now that AI can fill in the parts they aren't well-versed in. And that dramatically increases the chance of it happening. Being at risk of killing yours…

It sounds like your claim is "AI is a force multiplier, therefore it is dangerous and must be regulated." Not trying to strawman - I read your comment twice and I think that's what you're saying. By that logic Excel, the internet, calculators, and air conditioning are all force multipliers in that all of the make it easier for an expert who can make bioweapons able to do so more quickly / comfortably / easily. Obviou…

I didn't argue "must be regulated". I'm arguing AI is powerful (at achieving things, and hence has dual-use dangers). Many people won't even admit that, which is the part that really annoys me: they don't even want to have a conversation about risks because somehow AI is not actually a powerful general-purpose tool. Of course everything you said is true, though many of those things aren't very powerful. But the internet and social media and smartphones certainly have been of great utility for terrorism and child exploitation (and probably also causing many would-be-terrorists to get picked up).

Re: Pacing the frontier

#126

Post by soundworlds »

alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

It really comes down to this: Sam and Dario are leading companies that who made the reckless decision to train on everyone's IP without proper process or consent. It has been proven they trained on pirated content. There is no question about this.

Then, as soon as there are open models who threaten their businesses, they cry "distillation and IP theft".

It's the last 3 years of pure hypocrisy and fear-baiting for investment.

To be honest, I think people would now prefer the risk of misused open models, when the alternative is giving these guys more power.

Re: Pacing the frontier

#127

Post by dmarcos »

aesthesia wrote: post #103
dmarcos wrote: post #93

Earlier quoted context omitted.

None of the previous concerns were true and nothing indicates current ones are. It’s extrapolation, hypothetical apocalyptic imagined scenarios and piles of assumptions on how the technology could evolve. What makes even less credible in the AI case is that there’s financial / power to be gained by those making the claims. And to your point. Don Quixote, the first modern and greatest novel of all times, is about a gu…

I mean, Cervantes was parodying medieval romances. That's what Don Quixote was reading, not modern novels. The point is, it should give you at least some pause when the people most familiar with a technology are among those making the strongest statements about its potential dangers. Extrapolation has worked extremely well in predicting what AI will be capable of over the past 5 years; how confident are you that it w…

Cervantes was capturing the zeitgeist of the time: reading too many books can make you mad as they were becoming accesible to anyone. We know it’s not true.

People at anthropic don’t have any secret that many people outside already have. Again the burden of proof is on the person making a claim. That you feel concerned or scared doesn’t grant you special privileges to control or dictate a transforming technology.

Re: Pacing the frontier

#128

Post by pu_pe »

fidotron wrote: post #59
alach11 wrote: post #58

Earlier quoted context omitted.

I think some AI safety advocates have been arguing for great controls around nucleotide synthesis (e.g., impose greater data collection requirements or advanced screening for harmful sequences). It's surprisingly easy to order synthesized genes delivered to your doorstop. All you need is a credit card and a nucleotide sequence file! Developing a bioweapon is already within reach for experts with sufficient funds, tim…

> I think some AI safety advocates have been arguing for great controls around nucleotide synthesis (e.g., impose greater data collection requirements or advanced screening for harmful sequences). It's surprisingly easy to order synthesized genes delivered to your doorstop. All you need is a credit card and a nucleotide sequence file! So why have the discussion about regulating AI at all if that is the actual problem…

Because there are many legitimate uses for nucleotide synthesis, and we would have to slow scientists down by adding more regulation. The other problem is that it's not like it's very difficult to do it, so a motivated person could still do it even without access to these services.

Re: Pacing the frontier

#129

Post by padolsey »

fuddle wrote: post #3

> We request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development. This is quite vague to me, what exactly do you mean?

It means they haven't really thought it through and don't have a consolidated enough opinion to form a specific Ask.

Re: Pacing the frontier

#130

Post by txrx0000 »

NitpickLawyer wrote: post #118
alach11 wrote: post #26

The HN discourse on this subject is incredibly discouraging to me. It's completely nonconstructive. I take the pace of AI development seriously. All of my friends who work at AI Labs take it seriously. It's very easy to make lazy, shallow dismissals that leaders like Dario/Demis/Sam are acting purely out of self-interest. If AI does pose biological risk, or could destabilize geopolitics, or even just poses economic r…

> We should want the AI Labs to make binding agreements to proceed safely and slower That's a red herring, IMO. Once the cat is out of the bag, there's no "slowing down", only "slowing up". What's even more egregious is that the people behind the "slowing down" camp are also the people who've had 10s of billions invested in their companies. It smells of "we got ours, now you can't follow". It reeks of regulatory capt…

We have to stop comparing nuclear weapons with general intelligence. They are nothing alike.

Nukes are weapons of mass destruction that can only destroy or deter. Intelligence is dual-use/multi-use, with no restrictions on strategy. If we look at all organisms on Earth, species that are more intelligent are also more prosperous. This is true from bacteria, to insects, to animals, to humanity. Intelligence itself is not harmful.

There will be new adversarial games when everyone's smarter (bio/cyber offense/defense), but those games are always symmetrical in the long run, and there will also be new cooperative strategies. The reason some adversarial strategies appear to have an asymmetrical advantage right now is because we're too stupid, like how monkeys are too stupid to understand how humans work. Otherwise, we would not see more intelligent species being more prosperous as a general evolutionary trend.

Post reply on HN