AI generated code having more security issues + Superhuman AI security scanning capabilities could create a perfect storm.
One AI generates code, another AI finds the security holes and systems everywhere get compromised.
We have a year to fix security everywhere
351–360 of 373 posts
Re: We have a year to fix security everywhere
#352I'm confused why the worry about LLMs that will answer "how do I build a pipe bomb". That information is easily available other places. The anarchist cookbook has been around and available for 55 years, and yet pipe bombs are not going off all around us.
There used to be a thing called "Moore's Law of Mad Science": "Every eighteen months, the minimum IQ necessary to destroy the world drops by one point." Nowadays it is dropping much faster. At a certain point, the de-facto IQ needed to destroy the world will be low enough that someone can do it while they're having a psychotic break. There are millions of schizophrenics worldwide. Are you sure you want to roll those…
> "Every eighteen months, the minimum IQ necessary to destroy the world drops by one point."
That's a terrible framing. By this point it should be very easy for someone with 150 IQ but in reality they have issues finding someone to date.
Re: We have a year to fix security everywhere
#353[deleted]
Re: We have a year to fix security everywhere
#354Earlier quoted context omitted.
Who are these dedicated schizophrenics who are running long term super smart AIs to kill everyone without anyone noticing? Or are you implying that running LLM chatbots will give them this ability?
We're talking about open-weight models. It only takes one.
And hardware to run it and coherence and knowledge to set it up. This feels like insane doomerism.
Re: We have a year to fix security everywhere
#355I'm confused why the worry about LLMs that will answer "how do I build a pipe bomb". That information is easily available other places. The anarchist cookbook has been around and available for 55 years, and yet pipe bombs are not going off all around us.
There used to be a thing called "Moore's Law of Mad Science": "Every eighteen months, the minimum IQ necessary to destroy the world drops by one point." Nowadays it is dropping much faster. At a certain point, the de-facto IQ needed to destroy the world will be low enough that someone can do it while they're having a psychotic break. There are millions of schizophrenics worldwide. Are you sure you want to roll those…
Non-schizophrenic people have been responsible for more killings than schizophrenic people. Please be respectful and avoid stigmatising.
Re: We have a year to fix security everywhere
#356I'm confused why the worry about LLMs that will answer "how do I build a pipe bomb". That information is easily available other places. The anarchist cookbook has been around and available for 55 years, and yet pipe bombs are not going off all around us.
There used to be a thing called "Moore's Law of Mad Science": "Every eighteen months, the minimum IQ necessary to destroy the world drops by one point." Nowadays it is dropping much faster. At a certain point, the de-facto IQ needed to destroy the world will be low enough that someone can do it while they're having a psychotic break. There are millions of schizophrenics worldwide. Are you sure you want to roll those…
The minimum IQ necessary to swerve and kill a pedestrian is very low. Yet people don't do it :shrug:. It's almost like people don't want to destroy the world
Re: We have a year to fix security everywhere
#357Earlier quoted context omitted.
I have always hypothesised that AI is the great filter from the Fermi paradox. Given current velocity, AI will offer us cheap and abundant energy designs in a decade. The thing with cheap and abundant energy is that it can be used for good and bad. If nine billion people all receive access to plans to build a reactor which produces unlimited energy, it just takes one religious fanatic to end the world. And this is ju…
You’re leaving out some options for sure. Not everyone would need to live under the conditions of a police state, you could theoretically screen everyone and assign them to various levels of risk which would determine their level of supervision.
That just sounds like a police state with extra steps.
Re: We have a year to fix security everywhere
#358Earlier quoted context omitted.
We're talking about reporting breaches, not giving yourself some sort of abstract security score
And how do you know if you have been breached if you (negligently, in my opinion) have no audit logging, multiple principals sharing the same account, and no anomaly tracking? Does a breach only happen if the attacker brags openly about it? The difference with accounting is that, relatively speaking and certainly within this context, few businesses are cash businesses. Your bank is keeping at least a basic audit log…
forensic accounting and audits also require a paper trail.
So if you have no logging and such, you will have already failed regulatory reporting standards - just like you would fail an accounting audit if you have no paper trail of where your money went!
Re: We have a year to fix security everywhere
#359Earlier quoted context omitted.
Can you show me an example of a time that you prompted an LLM to provide some code, it did so, and then you were able to track down an original source for the output?
Every time I've asked an LLM to provide some code, it's given me an example from my own github repos. Granted, I've only asked it about some of the weirdly specific technological niche that my code inhabits.
Well that's funny, because you were making some pretty blanket statements upstream about how they "never" do this and "always" do that. And yet your experience seems extremely limited and niche, by your own admission.
Maybe you should experiment a little more. I think you will quickly learn that your previous impression is wrong. The days of them being merely some sort of jumped up autocomplete are years gone.
Re: We have a year to fix security everywhere
#360Earlier quoted context omitted.
I think that's separate. You can define an obvious risk e.g. "we may be infected with ransomware" and the security spending / productivity costs to stop it are still unlimited because nobody knows how to solve it.
You wouldn’t talk about ransomware like that for precisely the reasons you’d described: it’s a poorly defined open ended problem. You should tackle security in the same way you’d tackle any other kind of engineering initiative in IT. You break the problem down to identifiable tasks that can be easily marked as completed or not required (eg like developers track work in a KANBAN or sprint). So to take your ransomware…
Isn't that just sidestepping the issue? Setting up backups isn't a security task, it's just normal IT which businesses do indeed spend on because there are clear goals and predictable budgets. But just being able to restore data isn't the same thing as not getting ransomware. As you say, you can't define the latter as a goal exactly because it's a security goal, and so will turn into an infinitely long checklist of things you could potentially do with no guarantee of payoff.