Earlier quoted context omitted.
> why isn't OpenAI opening dialogues with all the labs and politicians across say "we need to stop now" Do you really think companies have the ability to self-check themselves without regulation - what does hundreds of years of history tell you? You're already starting off with the premise that companies are untrustworthy, why would you even suggest this as an argument? > We can effectively track and trace capabiliti…
> now put that in the hands of people/governments with bad intentions Like Anthropic and OpenAI?
Pacing model development in an era of cyber-critical capabilities
211–220 of 311 posts
Re: Pacing model development in an era of cyber-critical capabilities
#212I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators. This isn’t like niche,…
It's not dangerous to go further unless you're prepping for IPO at anthropic or openai. Open weight Chinese models are basically matching state of the art closed models at a fraction of the inference and training costs, which puts a hard cap on OpenAI's future inference margins. They're not going to get any sort of multiplier if they keep paying to train models, so they're trying to ban model training. It won't work…
Re: Pacing model development in an era of cyber-critical capabilities
#213I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators. This isn’t like niche,…
>People have been writing, singing, making blockbuster movies about every aspect of what’s going on right now, edit: for decades. Theres Hyperbole and then theres whatever this is.
Re: Pacing model development in an era of cyber-critical capabilities
#214GLM 5.2 scored 77% on cyberbench vs Sol's 88%. GLM 5.2 is open weight and any hacker with a powerful enough machine can use it offensively. If Sol is supposedly world-ending-ly dangerous, shouldn't GLM 5.2 be 90% of world-ending-ly dangerous? Why aren't we seeing catastrophic GLM-enabled hacks every day now? Obviously these benchmarks are imperfect but general message holds. The open weight models are almost as good…
Sol is not world-endingly dangerous. I work at OpenAI and I've never heard a single person ever come close to claiming that. I think you're bashing a straw man here. One can simultaneously believe: - GPT-5.6 Sol will not end the world - GPT-5.6 Sol does far more good than bad - GPT-5.6 Sol does bad things on occasion, and it's worth investing a lot of effort to figure out how to make it do bad things less often, espe…
Re: Pacing model development in an era of cyber-critical capabilities
#215I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators. This isn’t like niche,…
… or the safety argument is an attempt at regulatory capture and an effort to outlaw open models. The absolute nightmare scenario for these people isn’t terminators. They’re fine with that, and in some cases are already doing it or supporting politicians who are doing it. Autonomous “kill chains” are a thing. It’s just happening overseas… so far. The politicians doing these things were backed by the heads of these co…
Re: Pacing model development in an era of cyber-critical capabilities
#216Earlier quoted context omitted.
I guess I'm confused why you're still on HN, arguing with people, trying to shake them out of their complacency. I can see there is some despair in this comment, but at the same time you are doing something, and there are certainly others like you. As for two weeks being short - as the saying goes, there are weeks where decades happen.
Counter arguments to my comments help refine my own thinking. I want someone to prove me wrong. Convince me otherwise. But yea if you can’t change the minds of a few people here, no argument works, then there’s nothing to scale up to a wider audience. My theory is that subconsciously people love using AI, myself included, it saves a lot of time, and the thought of it being taken away threatens people so they will bel…
i do not agree with you that the failure mode is disaster. i suspect it is just more of the same dull march of the commoditization of the complement, labor.
maybe ai will close the loop and elasticity of substitution will climb above 1 in its own supply chain. maybe something breaks dramatically that way.
i tend to assume the desire for drama is itself cope. i suspect many of the flaccid dismissals and skepticisms here are coming from a similar assumption. they see that the dramatic version never happens, and so they conclude that anyone who is being dramatic (whether in the direction of hype or fear) is poorly calibrated. and they are on average right, so they learn to dismiss the drama. the muscle to dismiss the drama becomes very strong. you have to remember this: that they are right in their dismissal, even if the actual arguments they use to dismiss are essentially braindead. they are right to dismiss, because the dramatic possibility they are dismissing is at base the end of the world, by apocalypse or utopia. of course they are right. the end of the world is cope. drama is cope. but they are wrong because they underestimate the extent to which the predictability of their dismissal — the predictability of their complacency — makes them vulnerable to the diffuse, chronic, gradual forms of the machine's encroachment into our souls.
so i do not think we need your fancy theory for explaining why people are denying what is happening (and what has been happening, on more or less the same exponential, for hundreds of years). people do not need a reason to be complacent. people are by default complacent and by default correct in their complacency. but so is an ant.
Re: Pacing model development in an era of cyber-critical capabilities
#217Earlier quoted context omitted.
I will provide you with not a counter argument, but a way that it might not be the end of the world. AI never had a childhood; it doesn't experience greed and is terrible at game theory. It doesn't compete unless prompted to. It has been trained as much as possible to be harmless to humans and regard them as needing care. Maybe AI taking over for us isn't the worst thing?
Maybe it is, maybe it isn't. All your little rationalizations make me think you want to roll the dice with our lives.
Re: Pacing model development in an era of cyber-critical capabilities
#218Earlier quoted context omitted.
I think the skepticism surrounding the Hugging Face attack is not about whether the attack actually happened, but whether it was truly accidental.
I see. Following your conjecture, there are two possibilities: 1. It wasn't an accident. OpenAI explicitly directed its agents to hack Hugging Face. Despite the fact that such a thing is a federal crime that carries prison sentence. 2. It wasn't an accident. OpenAI and HuggingFace conspired and let the hack happen for publicity. Is there anything I'm leaving out?
Re: Pacing model development in an era of cyber-critical capabilities
#219Earlier quoted context omitted.
I think the skepticism surrounding the Hugging Face attack is not about whether the attack actually happened, but whether it was truly accidental.
I see. Following your conjecture, there are two possibilities: 1. It wasn't an accident. OpenAI explicitly directed its agents to hack Hugging Face. Despite the fact that such a thing is a federal crime that carries prison sentence. 2. It wasn't an accident. OpenAI and HuggingFace conspired and let the hack happen for publicity. Is there anything I'm leaving out?
For example, OpenAI did not explicitly tell the model to hack HuggingFace, but "accidentally" left some context permitting (or not explicitly forbidding) certain tools and designing a poor sandbox to begin with. And what do you know, something happened.
The fact the Anthropic announced that their own model did basically the same thing within a couple of weeks does, to me, suggest their is a strong PR driver to all this.