Everyone should slow down AI development except for me • Hacker News Board

Re: Everyone should slow down AI development except for me

#271

Post by ako » Sun, Sep 13, 2026, 8:54 AM UTC

teiferer wrote: post #229

aorloff wrote: post #123

Earlier quoted context omitted.

The very best outcomes involve us intentionally and collectively turning our attention away from this technology.

> The very best outcomes involve us intentionally and collectively turning our attention away from this technology. To expand on the "pure fantasy" sister comment: There is just no way this will happen. It's in the spirit of "we can just stop all wars" and "we can just end world hunger". Technically it's very easy to do. Socially it's impossible to do. Unless you ignore realities.

You forgot “we can stop climate change”…

Re: Everyone should slow down AI development except for me

#272

Post by ako » Sun, Sep 13, 2026, 8:55 AM UTC

carlob wrote: post #264

calgoo wrote: post #260

Earlier quoted context omitted.

The danger lies with someone asking the machine to solve those two questions and it decides to cheat the solution. Launching every nuke in the world stops all wars, just like wiping out 99% of humanity ends world hunger.

I don't think killing people solves world hunger, we already produce enough food to feed everyone, it's just not evenly distributed.

There can be more than one solution.

Re: Everyone should slow down AI development except for me

#273

Post by int_19h » Sun, Sep 13, 2026, 8:55 AM UTC

teiferer wrote: post #227

AngryData wrote: post #150

Earlier quoted context omitted.

Why? There are numerous technologies we have ignored or abandoned for numerous reasons. Many of the benefits of AI is "requires less humans", but in an age where we question what work people could possibly do in the future, human labor isn't that hard to find. If the amish can do what they do for whack religious reasons, people could manage it with ai for cultural and social reasons. And I never saw an amish person s…

> There are numerous technologies we have ignored or abandoned for numerous reasons. Are there? Nukes are a thing. Chemical weapons are a thing. Cluster bombs are a thing. Biological weapons are a thing. What is not a thing that shouldn't be? We say certain things should not be a thing, but then behind the scenes we made them a thing anyway.

On chemical weapons, most large state actors have got rid of them. Not because of any moral reasons, though, but simply because they don't actually work all that well in modern conventional warfare. This is then framed as an ethical issue, but you only need to look at where the same countries stand on e.g. landmines to realize how much bullshit it all is.

Conversely, where you do still see chemical weapons used, it's usually asymmetric conflicts where "just gas the rebels" actually works much of the time and is much cheaper than other options. Big guys can afford the other options though, someone like Assad, not so much.

More on this: https://acoup.blog/2020/03/20/collections-why-dont-we-use-ch...

Re: Everyone should slow down AI development except for me

#274

Post by matheusmoreira » Sun, Sep 13, 2026, 8:56 AM UTC

ChaoStatiX wrote: post #259

I really don't know what will happen with AI. It's honestly a bit frightening because the question I used to ponder just for fun as a kid 'What if AI revolts?'feels like it might not be that far off. It feels like ChatGPT came out just a few years ago, but seeing how far it has come already makes it impossible to predict what AI will be like in the future. I just hope humanity manages to navigate this well.

Yeah. I think it's the utter unreadability of the future that's most frightening. I used to have plans for the future... All replaced by anomie.

Re: Everyone should slow down AI development except for me

#275

Post by BoredomIsFun » Sun, Sep 13, 2026, 8:59 AM UTC

nullbio wrote: post #70

ranger_danger wrote: post #31

Earlier quoted context omitted.

I figured they're just admitting AI models have plateaued and are coming up with some fake story about self restraint so they don't lose VC money

The models have not plateaued, and they are not even mildly close to any sort of ceiling. Right now the barrier is data and compute. Quality data can be created synthetically at an exponential rate as models improve. Humans are actively feeding them with private IP. Compute advancements will begin to skyrocket as we unlock photonic computing and materials science advancements and scale up chip fabs. This is also comp…

> The models have not plateaued, and they are not even mildly close to any sort of ceiling.

Depends on defnition of "plateaued" and "ceiling". I am not impressed with 2026 consumer models at all.

> This is also compounding because the AI is accelerating the pace of research, testing, development, manufacturing, etc.

Yet it does accelerate - so is does Twitter. But does it to any substantial degree, esp. in AI theory? All the modern LLMs are the same old tired 2017 paper.

Re: Everyone should slow down AI development except for me

#276

Post by benfortuna » Sun, Sep 13, 2026, 8:59 AM UTC

ozgung wrote: post #238

AaronAPU wrote: post #27

They are talking about slowing down the public facing AI development. Because then nation states can create a capabilities gap between them and the public. Why does nobody seem to be pointing out this obvious explanation? It explains why the “we need to race China” concern suddenly vanished in the discussion. The government can simply gag Sam, Dario, Musk on national security basis, getting them all behind the public…

I’m pretty confident this has already happened. Public models are behind their internal models and just above Chinese models. Only thing closing the gap is Chinese models pushing. I’m not American. There is no way I use the same model as US Army for $20. When we have Sol they have Astra. When we have Astra they have Nova, Nebula, Galaxia…

At this point models are advancing way faster than we can figure out how to use them effectively. So having a generational advantage is not nearly as important as knowing how to use that advantage.

Re: Everyone should slow down AI development except for me

#277

Post by int_19h » Sun, Sep 13, 2026, 8:59 AM UTC

locknitpicker wrote: post #241

ben_w wrote: post #200

Earlier quoted context omitted.

Unclear. How much is China learning from open models? Or the other way around, how much is the US losing worldwide mindshare by refusing to allow non-Americans access to bleeding-edge models and letting China fill in the gap? (Playing catchup with Mythos is still catching up, etc.)

> How much is China learning from open models? The last batch of open weight models releases by Chinese companies are on par with the performance of US-based frontier models, even though they are designed to run on pretty unimpressive hardware.

This is very misleading. There is no Chinese model currently that is on par with Fable or Astra. Kimi K3 is pretty good, but it's not that good.

Furthermore the Chinese models in that class are appropriately large. K3, for example, is a 2.8T-parameter model. Qwen 3.8 Max, another comparable model, is 2.4T params. Even with MoE, these are not "designed to run on pretty unimpressive hardware". Stuff like Qwen3.7-27B is, but it is also not even in the same ballpark as Opus, never mind frontier.

Re: Everyone should slow down AI development except for me

#278

Post by int_19h » Sun, Sep 13, 2026, 9:01 AM UTC

0xcde4c3db wrote: post #64

lukan wrote: post #34

Earlier quoted context omitted.

Not sure about the level of irony here, but I keep hearing models have plateaued since a while now, but I keep being impressed with the latest model performance.

I don't think "plateaued" is the right word, but I do feel like there's been something like a logistic curve compression in the difference between smaller and larger models as the field evolves. For inference at least, the scale of practical difference between a single high-VRAM GPU or SFF UMA box, a whole rack, and a whole data center seems to be falling far short of what we might have imagined just a few years ago.…

The difference is still vast, it's just that the smaller stuff is "good enough" for many things now.

Re: Everyone should slow down AI development except for me

#279

Post by teiferer » Sun, Sep 13, 2026, 9:02 AM UTC

int_19h wrote: post #273

teiferer wrote: post #227

Earlier quoted context omitted.

> There are numerous technologies we have ignored or abandoned for numerous reasons. Are there? Nukes are a thing. Chemical weapons are a thing. Cluster bombs are a thing. Biological weapons are a thing. What is not a thing that shouldn't be? We say certain things should not be a thing, but then behind the scenes we made them a thing anyway.

On chemical weapons, most large state actors have got rid of them. Not because of any moral reasons, though, but simply because they don't actually work all that well in modern conventional warfare. This is then framed as an ethical issue, but you only need to look at where the same countries stand on e.g. landmines to realize how much bullshit it all is. Conversely, where you do still see chemical weapons used, it's…

So, you are actually arguing against yourself and are agreeing with me?

Re: Everyone should slow down AI development except for me

#280

Post by int_19h » Sun, Sep 13, 2026, 9:03 AM UTC

anon373839 wrote: post #194

rtpg wrote: post #183

Earlier quoted context omitted.

DGX Spark is a biiiiit costly but neat to hear! Do you know what kinda throughput you’re getting on that kinda setup? (I have a secondary problem of being “locked into” Claude Code by it being good enough for me, I’d probably need to investigate the other harnesses… my impression is other harnesses are a bit more aggressively OK with nuking your setup from orbit)

It is costly, especially right now. I don’t think you can make a case for it on cost savings! The throughput in a single stream is about 50 tokens/sec (a bit less for prose, a bit more for code due to speculative draft acceptance rates) and about 2,000 tokens/sec for prefill. Both numbers are flat and stable as context accumulates. That’s what finally tilted me away from the Mac Studio despite its much superior memor…

I don't think you can ever make a case for it on cost savings in general. Inference is very obviously the kind of problem where things are cheaper at scale, and this is still true for smaller models.

The only reason to run locally is privacy.