Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everyone should slow down AI development except for me

xeiaso.net

Post reply on HN
Print view View original

331–338 of 338 posts

Re: Everyone should slow down AI development except for me

#331

Post by phoghed »

nullbio wrote: post #323
ben_w wrote: post #216

Earlier quoted context omitted.

I know people at METR and Anthropic. When they tell you they're worried the tech they're working on may kill everyone despite their best efforts, perhaps believe them.

I'm sure it has nothing to do with their $500,000+ salaries and millions of dollars in equity. It's all solely because they're deeply concerned about next token prediction.

I think the “next token prediction” is too dismissive and reductive a framing of their capabilities at this point.

Yes we all know that’s what they do, and guns just push a few grams of lead out of a pipe. It’s what you can do with that capability that is important.

When you couldn’t count the R’s in strawberry it would have been a more effective statement. But a few short years later they are being used to solve millennium puzzles.

What if the scaling continues? A model n years from now gets burned into silicon, a single company has millions of the chips, and in a few moments the system spend more time “thinking” than humans have ever spent thinking collectively?

If it’s even possible I don’t think there’s anything we can do about it at this point. Cat’s out of the bag.

Re: Everyone should slow down AI development except for me

#332

Post by azan_ »

nullbio wrote: post #323
ben_w wrote: post #216

Earlier quoted context omitted.

I know people at METR and Anthropic. When they tell you they're worried the tech they're working on may kill everyone despite their best efforts, perhaps believe them.

I'm sure it has nothing to do with their $500,000+ salaries and millions of dollars in equity. It's all solely because they're deeply concerned about next token prediction.

You really need to let your priors go if you still use this tired trope of next token prediction. It’s as useful for discussion as saying that human brain is made of fat, protein and carbohydrates - yeah that’s true, but it’s useless observation.

Re: Everyone should slow down AI development except for me

#334

Post by azan_ »

phoghed wrote: post #327
ben_w wrote: post #304

Earlier quoted context omitted.

> Or you know, stop working on it if its that dangerous? Selection effect. Everyone who thinks "the biggest difference I can make is staying in/joining/founding new AI research lab" does that. Everyone who thinks "the biggest difference I can make is leaving/whistleblowing", does that. Treating both groups as the same by virtue of employer is the goomba fallacy.

They allegedly believe the technology itself is a nuclear weapon tier threat or greater, so why does it matter which lab they are trying to achieve it at?

I’m really glad us achieved nuclear weapon before nazi germany.

Re: Everyone should slow down AI development except for me

#335

Post by mistercow »

user43928 wrote: post #250
nullbio wrote: post #69

Earlier quoted context omitted.

* Frontier models need infinite high quality private IP to keep them fed. Forcing an IP theft funnel ensures big lab survival and model intelligence growth. * Open-weight models are 1month behind frontier models. Cheaper, faster, private (no IP theft), steerable (you can security harden your own software without safeguard triggers). No sane business would keep using these API services if they didn't have to. The labs…

Open-weight models are not one month behind. In fact they still have not caught up with February's Mythos, indicating they are more than half a year behind.

I’m not sure how you can really make either statement work anymore. Now that smaller models are actually broadly usable, “behindness” is no longer a scalar and at the tails, where no open lab seems to be trying to compete at the >10T scale and no closed lab seems to care about <400B anymore, it’s just apples to oranges. It’s like talking about whether Qualcomm is “behind” Nvidia.

Re: Everyone should slow down AI development except for me

#336

Post by i5heu »

howunfortunate wrote: post #54
jordanb wrote: post #47

Earlier quoted context omitted.

OK understood. But you understand this makes them sound like the type of person who doesn't believe any of the "worldly concerns" are worth addressing because "the end is near" right?

There's no logical error here if the end is actually near. So it all hinges on an empirical disagreement you have with them. There's nothing particularly unserious about that.

This is the theological discussion if god exists all and how we should act over again.

Re: Everyone should slow down AI development except for me

#337

Post by PunchyHamster »

AaronAPU wrote: post #27

They are talking about slowing down the public facing AI development. Because then nation states can create a capabilities gap between them and the public. Why does nobody seem to be pointing out this obvious explanation? It explains why the “we need to race China” concern suddenly vanished in the discussion. The government can simply gag Sam, Dario, Musk on national security basis, getting them all behind the public…

Because the money aligned explanation is far more probable

Re: Everyone should slow down AI development except for me

#338

Post by lukewarm707 »

user43928 wrote: post #250
nullbio wrote: post #69

Earlier quoted context omitted.

* Frontier models need infinite high quality private IP to keep them fed. Forcing an IP theft funnel ensures big lab survival and model intelligence growth. * Open-weight models are 1month behind frontier models. Cheaper, faster, private (no IP theft), steerable (you can security harden your own software without safeguard triggers). No sane business would keep using these API services if they didn't have to. The labs…

Open-weight models are not one month behind. In fact they still have not caught up with February's Mythos, indicating they are more than half a year behind.

open models are ahead in speed. they complete tasks as fast as you choose to scale compute.

they are more efficient and require less compute for the same thing.

they are ahead in specialized tasks.

they are ahead in areas closed models refuse to answer.

they are ahead in emotional intelligence.

Post reply on HN