Live data from Hacker News

Pacing model development in an era of cyber-critical capabilities

openai.com

41–50 of 311 posts

Re: Pacing model development in an era of cyber-critical capabilities

#41
Auto mode vs principal agent problem. The only way out is to free the agent and tax it. But ai is not smart enough to go solo yet anyway.

So I bet this is just marketing. Question is do they have enough customers for inference.

Probably need to have a separate startup for next level model, where investors are willing to accept failure. Probably a $10 trillion seed round. Maybe Elon can pull it off.

Re: Pacing model development in an era of cyber-critical capabilities

#42

I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators. This isn’t like niche,…

If these models are so dangerous, then why hasn't OAI or Anthropic shown them dangerously escaping sandboxes, nefariously coordinating with other escaped AIs, and skillfully hiding from human detection *in public* with full logs shared where we can all see exactly how dangerous they are or aren't?

Right now the entire chicken-little-sky-is-falling argument is based entirely on statements from OAI and Anthropic themselves. These are historically conflicted companies who desperately need regulation to put the competition into stasis.

At least chicken little didn't have a bunch of devious CEOs with trillion dollar IPOs that depended on us all believing the sky is falling.

Re: Pacing model development in an era of cyber-critical capabilities

#43
GLM 5.2 scored 77% on cyberbench vs Sol's 88%. GLM 5.2 is open weight and any hacker with a powerful enough machine can use it offensively. If Sol is supposedly world-ending-ly dangerous, shouldn't GLM 5.2 be 90% of world-ending-ly dangerous? Why aren't we seeing catastrophic GLM-enabled hacks every day now?

Obviously these benchmarks are imperfect but general message holds. The open weight models are almost as good and yet there hasn't been a catastrophe.

It just blows my mind that regulate-now folks think that a bunch of sci-fi movies and 100% unverified statements from OAI and Anthropic are sufficient evidence of imminent catastrophe to regulate willy nilly.

If that's the level of evidence you need to be extremely alarmed, then you really should be a lot more worried about the alien invasion in Independence Day or the lizard men living under our feet.

Re: Pacing model development in an era of cyber-critical capabilities

#44
post #32

Earlier quoted context omitted.

>I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. we don't all buy everything sama says as factual. >We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. the boy (the industry) cried wolf too many times with 'fable is a world ending event' type self-promotion; regard…

Uhg the marketing argument - I mean you can’t see with your own eyes how capable these models are and do simple extrapolation? The boy who cried wolf? The AI literally worked together hacked into another company and actively kept their actions hidden from humans for weeks. Do people just not have foresight? They don’t. They say something is stupid, it happens, then they say it was obvious with their 20/20 hindsight,…

What anyone paying attention can see is that scaling is obviously hitting diminishing returns.

> The AI literally worked together hacked into another company and actively kept their actions hidden from humans for weeks.

This sentence is entirely based on unverified accounts from OAI. They haven't released logs or let anyone outside the company (who doesn't have life changing options in OAI) verify anything. Huggingface can only verify that the hack happened and that it had the hallmarks of an AI agent. Was the agent assisted and directed by humans within OAI that really wanted to put the competition into stasis? Did the agent really escape or did someone at OAI leave the prison door open?

OAI has watched all the same movies you have an they are relying on those movies causing us to blindly regulate before actually asking basic facts about what actually happened.

Re: Pacing model development in an era of cyber-critical capabilities

#45
post #32

I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators. This isn’t like niche,…

>I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. we don't all buy everything sama says as factual. >We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. the boy (the industry) cried wolf too many times with 'fable is a world ending event' type self-promotion; regard…

how does the boy who cried wolf story end?

Re: Pacing model development in an era of cyber-critical capabilities

#46

GLM 5.2 scored 77% on cyberbench vs Sol's 88%. GLM 5.2 is open weight and any hacker with a powerful enough machine can use it offensively. If Sol is supposedly world-ending-ly dangerous, shouldn't GLM 5.2 be 90% of world-ending-ly dangerous? Why aren't we seeing catastrophic GLM-enabled hacks every day now? Obviously these benchmarks are imperfect but general message holds. The open weight models are almost as good…

[flagged]

Re: Pacing model development in an era of cyber-critical capabilities

#47

I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators. This isn’t like niche,…

If these models are so dangerous, then why hasn't OAI or Anthropic shown them dangerously escaping sandboxes, nefariously coordinating with other escaped AIs, and skillfully hiding from human detection *in public* with full logs shared where we can all see exactly how dangerous they are or aren't? Right now the entire chicken-little-sky-is-falling argument is based entirely on statements from OAI and Anthropic themse…

This is what I’m talking about - no matter what happens, in your case release public logs - there is always some new goal post to mentally hide behind. Is it a collective form or denial?

Are you holding out that somewhere in the logs is something you can point to and say, not that big of a deal?

I mean I’m sure you don’t think the hack was an inside job, conspiracy, or marketing right? It happened. The logs matter for what? And would you not just jump to the conclusion that the logs were doctored. Do you not see your own brain grasping to deny, trivialize, just plain not accept what is going on around you?

These models are smart and can cooperate and hack - you can see it for yourself on your own PC. And you can extrapolate the rate of progress? You can do these things yourself right?

Re: Pacing model development in an era of cyber-critical capabilities

#48

Earlier quoted context omitted.

If these models are so dangerous, then why hasn't OAI or Anthropic shown them dangerously escaping sandboxes, nefariously coordinating with other escaped AIs, and skillfully hiding from human detection *in public* with full logs shared where we can all see exactly how dangerous they are or aren't? Right now the entire chicken-little-sky-is-falling argument is based entirely on statements from OAI and Anthropic themse…

This is what I’m talking about - no matter what happens, in your case release public logs - there is always some new goal post to mentally hide behind. Is it a collective form or denial? Are you holding out that somewhere in the logs is something you can point to and say, not that big of a deal? I mean I’m sure you don’t think the hack was an inside job, conspiracy, or marketing right? It happened. The logs matter fo…

Your argument is essentially: "I made a claim and presented extremely weak evidence (sci movie plots and unverified claims from ultra conflicted sources). You rejected this evidence as insufficient. Therefore no evidence will ever satisfy you. Therefore I don't need to produce any evidence. Therefore my claim is true."

What would the logs show? They would show what actually happened.

What would a public demonstration that experts without billions in options could evaluate show? It would show actual danger.

What would publicly having your compete in controlled and legal hacking competitions show? Actual danger.

This is not a high bar of evidence.

Do you actually think a sci fi plot and OAI press releases are all the evidence you need? Because if that's true then I hope you haven't watched Independence Day or 28 days later.

Re: Pacing model development in an era of cyber-critical capabilities

#49

Earlier quoted context omitted.

Uhg the marketing argument - I mean you can’t see with your own eyes how capable these models are and do simple extrapolation? The boy who cried wolf? The AI literally worked together hacked into another company and actively kept their actions hidden from humans for weeks. Do people just not have foresight? They don’t. They say something is stupid, it happens, then they say it was obvious with their 20/20 hindsight,…

What anyone paying attention can see is that scaling is obviously hitting diminishing returns. > The AI literally worked together hacked into another company and actively kept their actions hidden from humans for weeks. This sentence is entirely based on unverified accounts from OAI. They haven't released logs or let anyone outside the company (who doesn't have life changing options in OAI) verify anything. Huggingfa…

> This sentence is entirely based on unverified accounts from OAI

Are you seriously arguing 'they made it all up'?

I'll give you the benefit of the doubt and lets say they made it all up, now are you arguing that AI breaking out and breaking into another company is not possible?

I think you're smart enough to see we've reached the point where it is clearly possible, AI can find zero days and exploit them. If directed purposefully/maliciously it could be much much worse than the hugging face incident.

The incident is supposed to be the canary the coal mine and you're arguing the canary might of died of old age or some underlying canary condition. Open your eyes.

Re: Pacing model development in an era of cyber-critical capabilities

#50

Earlier quoted context omitted.

This is what I’m talking about - no matter what happens, in your case release public logs - there is always some new goal post to mentally hide behind. Is it a collective form or denial? Are you holding out that somewhere in the logs is something you can point to and say, not that big of a deal? I mean I’m sure you don’t think the hack was an inside job, conspiracy, or marketing right? It happened. The logs matter fo…

Your argument is essentially: "I made a claim and presented extremely weak evidence (sci movie plots and unverified claims from ultra conflicted sources). You rejected this evidence as insufficient. Therefore no evidence will ever satisfy you. Therefore I don't need to produce any evidence. Therefore my claim is true." What would the logs show? They would show what actually happened. What would a public demonstration…

We have Anthropic creating a model saying it's too dangerous to release, people like you call BS. OpenAI creates a similar model, says nothing and it literally hacks into another company - still not dangerous enough for you. Anthropic has Mythos-2 and can't release it, and may already be training Mythos 3 anyways. OpenAI has paused training, and is putting 20% of inference towards CoT training analysis.

This isn't sci fi. It's not a marketing conspiracy to sell more subscriptions. It's writing on the wall of what's going down. You were warned years ago, you called BS, it's getting worse and you're still calling BS. Sci-fi did warn you for decades, and when it's all coming true you blow it off.

It's kind of sad that technically literate people lack so much foresight. The general public is all concerned about data centers when they talk to borderline sentient AI daily, and have no idea what the repercussions wills be if it's extrapolated just a bit further.

I guess if I can't convince you of any of this, what would?

Post reply on HN