Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pacing model development in an era of cyber-critical capabilities

openai.com

Post reply on HN
Print view View original

71–80 of 311 posts

Re: Pacing model development in an era of cyber-critical capabilities

#71

Post by KaiserPro »

kypro wrote: post #28
KaiserPro wrote: post #2

I used to work at a "frontier lab" before they were called such thing. We had three levels of lab isolation, one was basically a thin proxy to the internet. You were in a DMZ and that was about it. The next level was semi isolated, you were allowed some access to the internal network, but it was heavily firewalled, and you only had access to a limited number of internal services, and not internet. the last one was no…

If you look into what happened the details corroborated by hugging face make it seem extremely unlike to be deliberate or a "marketing ploy". People are just not taking any of this seriously enough. What happened was almost a textbook example of various risks AI doomers have been warning about for years. OpenAI's response? Pause training for 2 weeks. I mean we have senior people at these labs casually talking on podc…

> If you look into what happened the details corroborated by hugging face make it seem extremely unlike to be deliberate or a "marketing ploy".

I should clarify

There is a reason why we didn't have a artifact readthrough caching proxy in our system, because they are notoriously insecure. if you look that CVE history you can see its been full of bypass bugs for year. Also its not an isolated environment if you can arbitrarily pull through any package. If I was doing any kind of cyber training then any kind of unmonitored proxy would have been forbidden. Not because I am savant, but because I've seen what fuckery a human can get up to with the slightest hint of a proxy.

At best its negligence based on naïvety. the marketing around this is no mistake though.

Re: Pacing model development in an era of cyber-critical capabilities

#72

Post by bottlepalm »

mpalmer wrote: post #67
bottlepalm wrote: post #31

I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators. This isn’t like niche,…

You seem to be responding the way Mr Altman wants.

You seem to be in denial. Like really heavy denial. Alarm bells are going off everywhere. What people have been worried about for 100 years is actually happening. It's actually really obvious, but for some reason you're unable to fathom it.

In reality you're the one responding exactly how these big AI companies want - by not doing anything and letting them do whatever they want. For some reason telling you upfront it's dangerous only makes you more convinced that it's not.

Autonomously hacking out of training environments and into other companies by accident doesn't even trigger a response from anyone really. Crickets. I'm sure OpenAI themselves are amazed how little anyone cares.

Re: Pacing model development in an era of cyber-critical capabilities

#73

Post by reasonableklout »

bottlepalm wrote: post #70
reasonableklout wrote: post #36

Earlier quoted context omitted.

Regardless of the motivation, pausing training runs and reallocating compute to inference seem like a good move to me, and big news for the frontier. You also don't have to fully trust sama. There is plenty of pressure from internal employees and external (journalists etc.). It would be difficult for the company to take such a public position and simultaneously keep everyone quiet if it was a deception.

You see it as good news, I see it as writing on the wall that they are losing control. These actions won't scale for more powerful models. We knew the frontier was going to be dangerous, it is, it only gets more dangerous from here, and no one cares until it's too late.

Ok, but you still have two more weeks than you did before they paused the run. That's two more weeks for independent oversight, organizing politically, patching critical systems, or whatever you think is the right move, no?

Re: Pacing model development in an era of cyber-critical capabilities

#74

Post by bottlepalm »

reasonableklout wrote: post #73
bottlepalm wrote: post #70

Earlier quoted context omitted.

You see it as good news, I see it as writing on the wall that they are losing control. These actions won't scale for more powerful models. We knew the frontier was going to be dangerous, it is, it only gets more dangerous from here, and no one cares until it's too late.

Ok, but you still have two more weeks than you did before they paused the run. That's two more weeks for independent oversight, organizing politically, patching critical systems, or whatever you think is the right move, no?

Two weeks is a joke. The only ones happy are OpenAI’s competitors who now have two weeks to catch up.

I don’t know what the right move is - I see us driving down a road off a cliff, no exits, pedal glued to the floor.

Re: Pacing model development in an era of cyber-critical capabilities

#75

Post by vb-8448 »

bottlepalm wrote: post #31

I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators. This isn’t like niche,…

> because it’s literally getting dangerous to go further

What if there is no further at all?

Re: Pacing model development in an era of cyber-critical capabilities

#76

Post by anormalperson »

bottlepalm wrote: post #31

I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators. This isn’t like niche,…

There is an entire big world outside of Silicon Valley cults where literally no one gives a shit about AI prophecies. Shocking.

Re: Pacing model development in an era of cyber-critical capabilities

#77

Post by reasonableklout »

bottlepalm wrote: post #74
reasonableklout wrote: post #73

Earlier quoted context omitted.

Ok, but you still have two more weeks than you did before they paused the run. That's two more weeks for independent oversight, organizing politically, patching critical systems, or whatever you think is the right move, no?

Two weeks is a joke. The only ones happy are OpenAI’s competitors who now have two weeks to catch up. I don’t know what the right move is - I see us driving down a road off a cliff, no exits, pedal glued to the floor.

I guess I'm confused why you're still on HN, arguing with people, trying to shake them out of their complacency.

I can see there is some despair in this comment, but at the same time you are doing something, and there are certainly others like you.

As for two weeks being short - as the saying goes, there are weeks where decades happen.

Re: Pacing model development in an era of cyber-critical capabilities

#78

Post by anormalperson »

kypro wrote: post #28
KaiserPro wrote: post #2

I used to work at a "frontier lab" before they were called such thing. We had three levels of lab isolation, one was basically a thin proxy to the internet. You were in a DMZ and that was about it. The next level was semi isolated, you were allowed some access to the internal network, but it was heavily firewalled, and you only had access to a limited number of internal services, and not internet. the last one was no…

If you look into what happened the details corroborated by hugging face make it seem extremely unlike to be deliberate or a "marketing ploy". People are just not taking any of this seriously enough. What happened was almost a textbook example of various risks AI doomers have been warning about for years. OpenAI's response? Pause training for 2 weeks. I mean we have senior people at these labs casually talking on podc…

>People are just not taking any of this seriously enough.

What do you want us to do?

There is an obvious answer, and it was already the correct answer before we had LLMs: don't connect all your shit to the internet. That's it, that's literally it. We had a new invention, we went crazy with it for the past 30 years, and we connected everything, and now we will have to start thinking about what is actually worth connecting.

This is the debate we need to have.

Re: Pacing model development in an era of cyber-critical capabilities

#79

Post by joshstrange »

bottlepalm wrote: post #35
serf wrote: post #32

Earlier quoted context omitted.

>I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. we don't all buy everything sama says as factual. >We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. the boy (the industry) cried wolf too many times with 'fable is a world ending event' type self-promotion; regard…

Uhg the marketing argument - I mean you can’t see with your own eyes how capable these models are and do simple extrapolation? The boy who cried wolf? The AI literally worked together hacked into another company and actively kept their actions hidden from humans for weeks. Do people just not have foresight? They don’t. They say something is stupid, it happens, then they say it was obvious with their 20/20 hindsight,…

Do you know the story of the boy who cried wolf?

There may very well be a wolf lurking [0] but OpenAI/Anthropic have both cried wolf so many times, incorrectly, that it’s incredibly hard to believe “this time there IS a wolf!”. Remember “GPT-2 is too dangerous to release”?

I had a conversation at work just yesterday about how we need to start hardening things we’ve let languish because of the coming LLM-backed attacks we are sure to face, even if just from a script kiddy. I do think we are headed in that direction, however it’s Sam/Dario’s own fault that people aren’t going to take them seriously.

Lastly, as other have pointed out, this seems more financially motivated than our of any real desire for “safety”. We’ve all seen how both labs approach “safety” so it’s quite rich for them to now hide behind that after not giving a shit before.

[0] I don’t take anything Sam or Dario say at face value. The whole hacking thing could also be a case of them letting a model loose on purpose for the publicity, not an “escape” during a training run (or whatever they said). And when both, especially Sam, have lied so much and breathlessly warned about the dangers of AI (when it helped their bottom line and/or helped pull up the ladder behind them), it makes it hard to believe them.

Re: Pacing model development in an era of cyber-critical capabilities

#80

Post by insanitybit »

bottlepalm wrote: post #31

I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators. This isn’t like niche,…

It's not that dangerous, OpenAI just shit the bed building their infra. Write safer software and you'll be okay.
Post reply on HN