Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Pacing model development in an era of cyber-critical capabilities

openai.com

Post reply on HN
Print view View original

201–210 of 311 posts

Re: Pacing model development in an era of cyber-critical capabilities

#201

Post by bottlepalm »

hirvi74 wrote: post #184
bottlepalm wrote: post #145

Earlier quoted context omitted.

I am following the money, the money you, me, and everyone else is spending on AI. The money is telling me we are so dependent on AI now that we will say/think anything to tell ourselves that AI isn’t dangerous and any sign of danger is marketing. Either consciously or subconsciously you all are afraid of your favorite toy being taken away. You are all doing your collective part in spreading doubt about the warning si…

How dangerous can LLMs be in any immediate sense? I have a hard time feeling any existential dread from a threat that can be defeated by unplugging its servers. I believe the true threat is not LLMs. The true threat to humanity is the same as it has always been -- other humans.

What servers? There are thousands of datacenters around the world. Malicious AI leaks out, you'll never find it. You have the power to unplug stuff in your company, maybe country, but not internationally. Once you lose control it's gone.

Re: Pacing model development in an era of cyber-critical capabilities

#202

Post by protocolture »

bottlepalm wrote: post #115
nemomarx wrote: post #102

Earlier quoted context omitted.

Here is one thing I don't get - the model is only "running" if it's being kept going by some harness that is basically giving it prompts it's generating itself. Shouldn't kill switches be pretty easy to build into the software and hardware for this? and you would even have a better time dumping logs and analyzing things if you froze those processes any time something strange happened in testing, surely? So why do the…

> Shouldn't kill switches be pretty easy to build I need to remember when I comment here that these are the kinds of people I am replying to. Just oozing with hubris. No, kill switches are not easy to build and the latest incident should have made it clear that AI can go undetected, evade, zero day, and spread. The fact that this incident happened greatly increases the probability it happens again and/or is already h…

>I need to remember when I comment here that these are the kinds of people I am replying to.

People who dont buy into fantasism?

Re: Pacing model development in an era of cyber-critical capabilities

#203

Post by protocolture »

hirvi74 wrote: post #185
bottlepalm wrote: post #99

Earlier quoted context omitted.

Cool, well let me bring you up to date - it’s bad, and there’s no way to turn it off. Fiction has become non-fiction.

I wish people were this serious about real threats like climate change.

100% this line of thinking has a much better place to be.

Re: Pacing model development in an era of cyber-critical capabilities

#205

Post by bottlepalm »

hirvi74 wrote: post #186
bottlepalm wrote: post #115

Earlier quoted context omitted.

> Shouldn't kill switches be pretty easy to build I need to remember when I comment here that these are the kinds of people I am replying to. Just oozing with hubris. No, kill switches are not easy to build and the latest incident should have made it clear that AI can go undetected, evade, zero day, and spread. The fact that this incident happened greatly increases the probability it happens again and/or is already h…

> kill switches are not easy to build We've had circuit breakers for nearly a century.

And a circuit breaker has nothing to do with shutting AI off. What a strange comment.

Re: Pacing model development in an era of cyber-critical capabilities

#206

Post by bottlepalm »

Eisenstein wrote: post #179
bottlepalm wrote: post #85

Earlier quoted context omitted.

Counter arguments to my comments help refine my own thinking. I want someone to prove me wrong. Convince me otherwise. But yea if you can’t change the minds of a few people here, no argument works, then there’s nothing to scale up to a wider audience. My theory is that subconsciously people love using AI, myself included, it saves a lot of time, and the thought of it being taken away threatens people so they will bel…

I will provide you with not a counter argument, but a way that it might not be the end of the world. AI never had a childhood; it doesn't experience greed and is terrible at game theory. It doesn't compete unless prompted to. It has been trained as much as possible to be harmless to humans and regard them as needing care. Maybe AI taking over for us isn't the worst thing?

Maybe it is, maybe it isn't. All your little rationalizations make me think you want to roll the dice with our lives.

Re: Pacing model development in an era of cyber-critical capabilities

#207

Post by gbnwl »

thomascountz wrote: post #121
bottlepalm wrote: post #31

I don’t get how this is not the top post on HN. This should be like alarm bells going off, canary in the coal mine type of stuff. We’re hitting the frontier of the frontier where we can’t go further because it’s literally getting dangerous to go further. And meanwhile somehow this lack of concern mirrors the real world where normal people are more concerned about data centers than terminators. This isn’t like niche,…

Follow the money: who told you that OpenAI's models autonomously coordinated to hack external systems? What incentives might they have to want you to believe that story? Are there priors which demonstrate them benefiting from telling similar stories, regardless of their factuality? But to your counterpoint, let's say the story is 100% true, because I agree it is at least plausible. What would the incentive be for the…

It seems like the entire thought process you’re trying to sell hinges on the idea that OpenAI reported the attack first. Did you forget that it was actually HuggingFace that reported it first, and OpenAI only stepped forward latter?

Re: Pacing model development in an era of cyber-critical capabilities

#208

Post by XMPPwocky »

solid_fuel wrote: post #197
XMPPwocky wrote: post #195

Earlier quoted context omitted.

> My point here is there is no continuous state that is not computed from the context. Oh, are you talking more about the lack of continual learning across context windows? Gotcha if so, my error. Could you explain why running without sensory input is relevant here? It strikes me as unrelated to how dangerous/hard-to-"kill" something is (sure, I could run without sensory input, but I'm not doin' anything anymore!) -…

> Oh, are you talking more about the lack of continual learning across context windows? Gotcha if so, my error. Sort of. I'm talking about the lack of recurrence specifically. In nature, brains are recurrent - they are full of loops where internally computed state is looped back into the network at a "previous" layer (brains are not strictly layered like our machine imitations of them are). This is in contrast to LLM…

Thanks for the response! This helps me understand what you meant way better. I'm about to go to bed here, but I'll respond in the morning. Ty!

Re: Pacing model development in an era of cyber-critical capabilities

#209

Post by bottlepalm »

skydhash wrote: post #182
bottlepalm wrote: post #170

Earlier quoted context omitted.

The fact you posted that and nothing of substance tells me you have nothing, or something very weak. So please tell me of this magical unhackable software/hardware you vague post about.

You're the one that is inventing this "magical unhackable software/hardware", the other person was just saying that there are ways to write "safer" software, not "safe" software. Anything that has happened looks like no security concerns has been looked at or have been thought about.

'Safer software is meaningless when it comes to SOTA AI. If it can be hacked it will be, quickly. This isn't the old days with a finite number of human hackers that need food and sleep to keep hacking.

Therefore security becomes binary. It is either perfect or it isn't. If there there is the slightest mistake anywhere AI will find it and carve it up. My point is obviously perfect software doesn't exist. The malicious AI gets out, literally turns everything inside out and locks you out of your car, computer, phone, office, the airplanes don't fly anymore. I don't know what to tell you. Computer security is on the brink of basically not existing as you know it with the bar being literal perfection.

Re: Pacing model development in an era of cyber-critical capabilities

#210

Post by bottlepalm »

insanitybit wrote: post #169
bottlepalm wrote: post #150

Earlier quoted context omitted.

People make mistakes, and people don’t know everything either. The software you write is on top of a house of cards of software and hardware. It all has to be perfect to not be hacked. It isn’t perfect, even if you try your hardest it won’t be perfect and to argue it’s not difficult is absurd. You don’t know everything, you don’t own the stack. So how are you going to create a secure anything top to bottom - you can’…

> It all has to be perfect to not be hacked. This is absolutely not true. It's a matter of cost. Exploitation can cost on the order of 10K, 100K, 1M, 10M, etc. A straightforward one would be something like "MD5 collisions are on the order of $100K-1M" (a while ago, at least) so if you used MD5 you knew that it costs about that much to bypass the control. Moving to SHA1 pushes you massively out of that space, even if…

It just takes one crack in the armor, and malicious AI has the potential to exploit it faster than you have time to react. Literally go to bed and wake up locked out of everything with no hope of recovery.
Post reply on HN