Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pacing model development in an era of cyber-critical capabilities

openai.com

Post reply on HN
Print view View original

301–310 of 311 posts

Re: Pacing model development in an era of cyber-critical capabilities

#301

Post by protocolture »

solid_fuel wrote: post #299
protocolture wrote: post #293

Earlier quoted context omitted.

>copying itself to as many data centers as it can So this is the part thats never happened, and is extraordinarily unlikely to occur. A "Datacentre" isnt a big box with "Insert AI here" on the side.

It's pretty funny to watch people look at these things - running billions of weights on custom cerebras hardware in dedicated datacenters the size of a city block, pulling 10's of megawatts - and panic that it's just going to copy itself into AWS. It just speaks to a fundamental ignorance of what an LLM is, how large the big hosted ones are, and the software architecture that makes it all work.

I feel like someones going to try and write a skill file to accomplish this and its going to be way too hard.

Re: Pacing model development in an era of cyber-critical capabilities

#302

Post by protocolture »

bottlepalm wrote: post #232
protocolture wrote: post #199

Earlier quoted context omitted.

>Breaking out? It didnt break out in any meaningful sense. What it did was get access to the internet. You take it as granted that there was anything meaningful there to stop it. But heres the kicker, they have been testing these things connected to the internet anyway. What it did was get a level of access it has otherwise been granted in other simulations. Its not exactly the same as any of the scifi AI breakout sc…

I'm sorry my jaw is on the floor reading this complete disregard of AI literally not only escaping containment, twice, but then infiltrating another company with multiple zero day attacks going undetected for great lengths of time. The plausible sci-fi scenario from here is obvious. Intentionally bad, or unintentionally bad AI zero days as much as as it can, as fast as it can, copying itself to as many data centers a…

>I think we all know none of it is very secure software wise against a SOTA AI that can literally come up with its own zero day attacks.

I mean this bits rich too, it demonstrates a pretty poor understanding of modern security practices.

Like having a single element of perimeter security is like 1990s security. We do defense in depth these days. Not to mention multiple overlapping controls for every element.

It has been tested against AI bro security and found it wanting. Extrapolating that to the every system on the planet is nutso bananas.

Actually if you think anything works this way just tell an AI model to go fetch you some money from the bank. Assuming it gets anywhere close to achieving its goal you can get an object lesson in modern SIEM processes when the feds explain the charges and evidence.

Or maybe theres another reason my phone blows up when someone so much as edits a config file on a protected system.

Re: Pacing model development in an era of cyber-critical capabilities

#303

Post by protocolture »

bottlepalm wrote: post #205
hirvi74 wrote: post #186

Earlier quoted context omitted.

> kill switches are not easy to build We've had circuit breakers for nearly a century.

And a circuit breaker has nothing to do with shutting AI off. What a strange comment.

It literally does.

Re: Pacing model development in an era of cyber-critical capabilities

#304

Post by protocolture »

bottlepalm wrote: post #209
skydhash wrote: post #182

Earlier quoted context omitted.

You're the one that is inventing this "magical unhackable software/hardware", the other person was just saying that there are ways to write "safer" software, not "safe" software. Anything that has happened looks like no security concerns has been looked at or have been thought about.

'Safer software is meaningless when it comes to SOTA AI. If it can be hacked it will be, quickly. This isn't the old days with a finite number of human hackers that need food and sleep to keep hacking. Therefore security becomes binary. It is either perfect or it isn't. If there there is the slightest mistake anywhere AI will find it and carve it up. My point is obviously perfect software doesn't exist. The malicious…

> If it can be hacked it will be, quickly.

No lol. AI can locate software vulns, but it doesn't have any magical tricks that let it overcome 3 or 4 safeguards at once.

I mean it has the same knowledge gaps as anyone else. It doesn't even know what services to subvert before it attempts access triggering threat response. It cant find 5 zero days simultaneously without connecting to those ports for service identification and generating alerts. You just keep exposing your staggering lack of technical understanding.

You remind me of this kid in high school who claimed he could hack my computer and he had my IP address, when I knew for a fact I had Dial Up which was normally disconnected, and when it wasnt it sat behind CGNAT and there was piss all he could do.

I should mention that, PCI DSS mandates logging, alerts, overlapping controls. These are very widely implemented best practices. They might not be present in some AI bro startups, but anything that matters generally looks like this.

Re: Pacing model development in an era of cyber-critical capabilities

#306

Post by bottlepalm »

solid_fuel wrote: post #297
bottlepalm wrote: post #231

Earlier quoted context omitted.

The knife is inanimate. The LLM is not. OpenAI prompted some employee to run the tests. The employee prompted the LLM. The LLM setup a message board and prompted other LLMs, and the fly wheel was running. It had to be turned off manually otherwise it'd still be going today. It's funny how a year ago talking about this kind of stuff would be laughed at by people like you, saying, "it's never happened before". Well it…

> The knife is inanimate. The LLM is not. Put an LLM on your GPU. Give it no prompt. What happens? nothing This is because LLMs are inanimate, just like a knife. Just like a gun.

Put a person in a room, give them no food, what happens? Inanimate.

Re: Pacing model development in an era of cyber-critical capabilities

#307

Post by bottlepalm »

solid_fuel wrote: post #295
zmgsabst wrote: post #239

Earlier quoted context omitted.

I agree with you about which objects are motive, ie, LLMs do just sit there unprompted. > I believe that this recurrence is where "intelligence" lives - and I believe it is the difference between a thinking being and a stochastic parrot. My objection was to this, on technical grounds: LLMs exhibit intelligence. 1. They reason in an internal type theory. 2. This type theory is meaningfully encoded from the actual data…

> Intelligent and reasoning doesn’t entail self-motive I disagree. I believe that LLMs do exhibit reasoning , but not intelligence . A simple dictionary definition of intelligence from duck duck go is "the ability to acquire, understand, and use knowledge." LLMs can reason using the knowledge they already possess, but they cannot of their own accord decide to go out and acquire new knowledge. Not without being prompt…

Your entire life can represent a short term context. It goes away when you do. You use your life time add to your context to do/learn whatever, but it's finite, and it ends.

If you weren't prompted by your parents, schools, teachers. Then you wouldn't be acquiring knowledge either. The most you'd be acquiring without that advanced prompting would be bugs in the woods to eat.

Re: Pacing model development in an era of cyber-critical capabilities

#308

Post by solid_fuel »

bottlepalm wrote: post #307
solid_fuel wrote: post #295

Earlier quoted context omitted.

> Intelligent and reasoning doesn’t entail self-motive I disagree. I believe that LLMs do exhibit reasoning , but not intelligence . A simple dictionary definition of intelligence from duck duck go is "the ability to acquire, understand, and use knowledge." LLMs can reason using the knowledge they already possess, but they cannot of their own accord decide to go out and acquire new knowledge. Not without being prompt…

Your entire life can represent a short term context. It goes away when you do. You use your life time add to your context to do/learn whatever, but it's finite, and it ends. If you weren't prompted by your parents, schools, teachers. Then you wouldn't be acquiring knowledge either. The most you'd be acquiring without that advanced prompting would be bugs in the woods to eat.

Ah, you have some serious gaps in your understanding of how human minds work. Life experiences are not just a "short term context" - they are used to actually refine the network within the human brain. Your life experiences do not just live in a single stream with access mediated by attention.

Please see https://en.wikipedia.org/wiki/Memory as a starting point. I recommended reading through some information on development psychology too, that will help clear up these misunderstandings. Hope that helps!

Re: Pacing model development in an era of cyber-critical capabilities

#309

Post by bottlepalm »

solid_fuel wrote: post #308
bottlepalm wrote: post #307

Earlier quoted context omitted.

Your entire life can represent a short term context. It goes away when you do. You use your life time add to your context to do/learn whatever, but it's finite, and it ends. If you weren't prompted by your parents, schools, teachers. Then you wouldn't be acquiring knowledge either. The most you'd be acquiring without that advanced prompting would be bugs in the woods to eat.

Ah, you have some serious gaps in your understanding of how human minds work. Life experiences are not just a "short term context" - they are used to actually refine the network within the human brain. Your life experiences do not just live in a single stream with access mediated by attention. Please see https://en.wikipedia.org/wiki/Memory as a starting point. I recommended reading through some information on develo…

By that logic engineers don't understand flying because planes don't flap their wings like birds. Not everything has to work the same to meet your arbitrary criteria for intelligence and reasoning.
Post reply on HN