Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pacing model development in an era of cyber-critical capabilities

openai.com

Post reply on HN
Print view View original

101–110 of 311 posts

Re: Pacing model development in an era of cyber-critical capabilities

#101

Post by bottlepalm »

XorNot wrote: post #95
scarmig wrote: post #54

Earlier quoted context omitted.

https://en.wikipedia.org/wiki/Don%27t_Look_Up A movie fit for our time. You can produce detailed descriptions of the incident, verified by adversarial parties, and some people will still scream "it's a conspiracy! It's a marketing stunt!" This is all very unfortunate--there's a meaningful chance that AI will cause unprecedented disaster, with the HF incident being just a small preview, but people would rather squawk…

In don't look up anyone with a telescope could've confirmed the danger. Hence the title. In the real world, absolutely no one except a bunch of heavily fiscally incentivized parties with unclear relationships are saying anything happened. The subsequent dog pile of other companies to say "they were near the AI hacking too!" should make you even more suspicious: Anthropic jumped in and why was Tailscale posting about…

The progress and increase in capabilities is undeniable. You’re blind if you don’t see where it’s heading.

Re: Pacing model development in an era of cyber-critical capabilities

#102

Post by nemomarx »

bottlepalm wrote: post #99
nemomarx wrote: post #90

Earlier quoted context omitted.

Most of that concern was in fiction. Non theoretical, genuine concern about AI is pretty recent, maybe dating back to 2010 ish with the rationalist types. But also no one trusts anyone involved in AI safety now, I think, because they are all seemingly in bed with these big companies. And there is the perpetual argument "if we aren't pushing AI forward China will and then we don't have any control" and so on. (I don't…

Cool, well let me bring you up to date - it’s bad, and there’s no way to turn it off. Fiction has become non-fiction.

Here is one thing I don't get - the model is only "running" if it's being kept going by some harness that is basically giving it prompts it's generating itself. Shouldn't kill switches be pretty easy to build into the software and hardware for this? and you would even have a better time dumping logs and analyzing things if you froze those processes any time something strange happened in testing, surely?

So why do the big frontier labs not have something like this anyway. They're talking about two week pauses on the new model (which seems very short and hardly a cost at all to me) and alarms during their tests that might be 30 minutes late and etc. Those are not very serious measures, so are they not concerned?

Re: Pacing model development in an era of cyber-critical capabilities

#103

Post by bottlepalm »

alehlopeh wrote: post #89
bottlepalm wrote: post #72

Earlier quoted context omitted.

You seem to be in denial. Like really heavy denial. Alarm bells are going off everywhere. What people have been worried about for 100 years is actually happening. It's actually really obvious, but for some reason you're unable to fathom it. In reality you're the one responding exactly how these big AI companies want - by not doing anything and letting them do whatever they want. For some reason telling you upfront it…

I don’t understand. What does it matter how many years people have been worrying about it?

Because it shows this very course of events has been thought of over and over again for decades. I’m not making this up, and my reaction is natural.

Your reaction to everything happening is the unnatural one, and your complacency would fit perfectly into a comedy/tragedy story regarding the rise of AI.

I can imagine how many of our ancestors that warned us are turning in their graves right now observing our reactions calling this stuff marketing. It’s idiocracy.

Re: Pacing model development in an era of cyber-critical capabilities

#104

Post by rubendev »

kalkin wrote: post #96
insanitybit wrote: post #80

Earlier quoted context omitted.

It's not that dangerous, OpenAI just shit the bed building their infra. Write safer software and you'll be okay.

All we need to retain human control over AIs is for nobody to write any bugs. Piece of cake.

No we just need developers to do the bare minimum of effort to write secure software. Most hacks are not super complicated vulnerabilities chained together, but just utter failures where authentication and authorization was simply forgotten or untested, or where nobody bothered to validate the data they receive.

The bar for software is so low that it is embarrassing for the entire profession.

Re: Pacing model development in an era of cyber-critical capabilities

#105

Post by vb-8448 »

bottlepalm wrote: post #83
vb-8448 wrote: post #75

Earlier quoted context omitted.

> because it’s literally getting dangerous to go further What if there is no further at all?

[flagged]

I don't know if it's a fantasy or not, but asking if there is limit on what can be achieved is legit question.

Re: Pacing model development in an era of cyber-critical capabilities

#106

Post by reducesuffering »

bottlepalm wrote: post #31

I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators. This isn’t like niche,…

It is extraordinarily emotionally hard for someone to stare down the terrible implications of what is unfolding. All manner of rationalization and cope will be applied to come up with excuses; motivated reasoning.

The CIA director will call AGI capabilities "digital nuclear weapons" and Geoffrey Hinton will estimate a 50% probability AGI ends humanity, and half of HN will call every new evidence of disaster a marketing stunt.

Re: Pacing model development in an era of cyber-critical capabilities

#107

Post by bottlepalm »

mrguyorama wrote: post #98
bottlepalm wrote: post #84

Earlier quoted context omitted.

Has the last 100 years of concern about AI and robots been crying wolf because it hasn’t happened yet? How does reallocating resources from training to chain of thought monitoring make ‘financial’ sense? You suggesting then model was let loose on purpose.. how am I the crazy one here while all of you are pushing this tin foil hat conspiracy angle?

You know fiction is.... not real, right? We have several films about the sun or earth needing to be restarted with a nuclear weapon. That doesn't make it something we should be concerned about. Hell, half the fiction about evil AI is actually commentary on stuff that already exists and is making us suffer and doesn't have anything to do with any potential future AI The Star Trek TNG episode about Data being tried in…

Your argument boils down to some sci-fi is unrealistic therefore all of it is.

Sci-fi is supposed to make you think. What if the AI told you NO when you need it - Hal 9000. What if the evil AI got out and you don’t know what data center it’s hiding in - Lawnmower Man. What if you were so sure something couldn’t escape but it did - Jurassic Park.

Actually all three of those predicted fantastical scenarios are possible today. So what’s next? Don’t stick your head in the sand - I assure you the next disaster has already been predicted, and I’m sure if you think about it a little you can figure out what it is.

Re: Pacing model development in an era of cyber-critical capabilities

#108

Post by thoughtpeddler »

tedsanders wrote: post #56
red_green_yell wrote: post #43

GLM 5.2 scored 77% on cyberbench vs Sol's 88%. GLM 5.2 is open weight and any hacker with a powerful enough machine can use it offensively. If Sol is supposedly world-ending-ly dangerous, shouldn't GLM 5.2 be 90% of world-ending-ly dangerous? Why aren't we seeing catastrophic GLM-enabled hacks every day now? Obviously these benchmarks are imperfect but general message holds. The open weight models are almost as good…

Sol is not world-endingly dangerous. I work at OpenAI and I've never heard a single person ever come close to claiming that. I think you're bashing a straw man here. One can simultaneously believe: - GPT-5.6 Sol will not end the world - GPT-5.6 Sol does far more good than bad - GPT-5.6 Sol does bad things on occasion, and it's worth investing a lot of effort to figure out how to make it do bad things less often, espe…

What do you recommend people who are technically inclined enough to participate meaningfully here on HN, but do not work at the labs and cannot assist in that capacity, do to help the broader public understand this technology better and mitigate potential risks (by e.g. ‘up-leveling everybody’ through AI literacy etc and other sorts of collective defensive efforts)?

Re: Pacing model development in an era of cyber-critical capabilities

#109

Post by bottlepalm »

kalkin wrote: post #93
serf wrote: post #32

Earlier quoted context omitted.

>I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. we don't all buy everything sama says as factual. >We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. the boy (the industry) cried wolf too many times with 'fable is a world ending event' type self-promotion; regard…

> fable is a world ending event Did anyone actually say this? Mostly what I have seen is people saying "hey at some point these models might get dangerous." And the type of HN commenter who mistakes blind cynicism for wisdom laughs that off as marketing. And now when (some) worries appear to come true, somehow having previously expressed those worries is not being proved right, but in fact discrediting, because it wa…

The world ending stuff is a straw man HN readers have created to point and laugh at. Laughing helps mask the underlying concern.

Re: Pacing model development in an era of cyber-critical capabilities

#110

Post by rubendev »

red_green_yell wrote: post #42
bottlepalm wrote: post #31

I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators. This isn’t like niche,…

If these models are so dangerous, then why hasn't OAI or Anthropic shown them dangerously escaping sandboxes, nefariously coordinating with other escaped AIs, and skillfully hiding from human detection *in public* with full logs shared where we can all see exactly how dangerous they are or aren't? Right now the entire chicken-little-sky-is-falling argument is based entirely on statements from OAI and Anthropic themse…

I think the model was able to escape the sandbox and hack huggingface because they were incompetent or not giving enough priority to implementing basic cybersecurity principles.

If they would have done so, there wouldn’t have been an escape or a hack. The reason we don’t get much details is because the details are embarrassing for them.

Post reply on HN