It's of course important to reduce the impact surface before more powerful models become available, but it's worrying that some people seem to assume the only way to counteract those are by using more AI models, which despite being (apparently) good at finding bugs, they are also very good at introducing them unnoticed.
We have a year to fix security everywhere
281–290 of 373 posts
Re: We have a year to fix security everywhere
#282I'm confused why the worry about LLMs that will answer "how do I build a pipe bomb". That information is easily available other places. The anarchist cookbook has been around and available for 55 years, and yet pipe bombs are not going off all around us.
Some of the recipes in the Anarchist Cookbook are incorrect. I know, I had a copy from Paladin Press.
Re: We have a year to fix security everywhere
#283> On September 22, Apple is releasing the M5 Mac Studio with 256 GB of unified memory [..] it will probably [..] enough to write this snippet of code in 3 seconds The author has obviously never ran an LLM on a mac! In 3 seconds, it will have possibly started to think about maybe scheduling a date to contemplate the planning timeline for processing the second token in your prompt.
Also, LLMs never *write* code snippets, they just pirate them from somewhere else.
News: "LLM Models might kill us all!..."
HN user: "I take issue with the precise definition of one word in the article..."
Re: We have a year to fix security everywhere
#284Earlier quoted context omitted.
A large number of places will buy a new firewall every 5 years, or pay their fortinet renewal and check "Security: Done!" without any kind of analysis. I was contracted in to a place to do among other things cyber security insurance audits, and they asked me to stop doing them because I refused to lie to their insurer. "Wait but if we only score 20 / 300 that makes us look kind of bad" uh huh.
> pay their fortinet renewal and check "Security: Done!" without any kind of analysis. there exists objective measure of security, which would be some sort of hacks/breaches per period. If customers cared about it (and i assume they do), they would choose companies that have less breaches over others with higher counts, normalized on cost differences. Therefore, if companies didnt actually try to fix their security b…
The number of breaches would have to be honestly reported for that idea to work. None of the security firms would want to do that; least of all the lowest quartile of them.
Re: We have a year to fix security everywhere
#285Earlier quoted context omitted.
> I don't see how dumb entity can align smart entity. And we are dumb ones. Super intelligence will play aligned until it is not, and then it'll strike. I am also worried about this. One ray of sunshine here is that it's not a great explanation for the Fermi paradox. We would see AI civilizations all over the galaxy if this were a regular occurance. I guess that just leaves us with the "horrifying" scenario: Geoffrey…
> For example, in order to maximise aggregate wellbeing, it would be necessary to hurt individuals. Executing people who are burdensome to society. [citation needed] With all due respect, I think this statement is false. If you live in a society that can, thanks to advanced technology, provide for the material needs of all, which, if you live in a rich Western country, is basically already the case (remember the last…
I'll use a very specific example to explain my point, because I don't disagree with what you write: psychopaths. Psychopaths are 20x more prevalent among homicide offenders. Removing them from society would significantly reduce the number of homicides. It would make their friends and family sad, but this is a much smaller aggregate impact than a homicide.
This also illustrates the messy nature of comparing social good and bad. Who am I to argue that the aggregate sadness created by removing a psychopath from society is better than a homicide? Maybe the person murdered was disliked by everyone and had no friends?
I remain resolute in my premise, however, and there is a lot of theory on the issues of aggregate social good vs individualism. John Stuart Mill's On Liberty (1859) is an excellent foundation for this. This is a subjective problem to solve, meaning that the balance between individual freedom and aggregate wellbeing is different for everyone. An AI overlord would make decisions every day which individuals would disagree with, at least at the margins.
Re: We have a year to fix security everywhere
#2861 year left for cybersecurity hardening, I thought so as well. The issue is, even if we get it done: in 1 year the models will be so good in social-engineering that they will be able to extract any information they want anyway. Happy to be falsified here, if anyone has evidence-based arguments. EDIT: by social-engineering I mean for example: recon company structures, gathering and merging people's data from the dark-…
I have to agree. People are worried about WordPress. We should be talking about scripts as sophisticated as Shattered Spider targeting every ISP. In an environment where IT has to wade through vendor chaos.
Re: We have a year to fix security everywhere
#287Earlier quoted context omitted.
I have always hypothesised that AI is the great filter from the Fermi paradox. Given current velocity, AI will offer us cheap and abundant energy designs in a decade. The thing with cheap and abundant energy is that it can be used for good and bad. If nine billion people all receive access to plans to build a reactor which produces unlimited energy, it just takes one religious fanatic to end the world. And this is ju…
> The thing with cheap and abundant energy is that it can be used for good and bad. Yes, but the more important thing is the imbalance. So-called "AI" can be used far more effectively and efficiently for bad. > I have come to the conclusion that we should not allow everyone access to unlimited intelligence. You meant unlimited information, right?
No, intelligence. We already have unlimited information (not to be confused with omniscience). The application thereof is the gate right now between today and planet-scale destructive weaponry. At some point it becomes an academic distinction because this God-like intelligence could provide very easy to follow plans to cause planet-scale destruction, and you could call that "information." The predicate is the intelligence and how it's used.
Re: We have a year to fix security everywhere
#288More tired “don’t use C or C++” advise.
I wonder why C and C++ are usually regarded as equally insecure. In C you need to carefully check that you free allocated memory, and that you don't use it after you free it. In C++ this is automated by using classes like std::string and std::vector, once they go out of scope their memory is freed and you can't use it anymore. It is still possible, e.g. by using a for loop that iterates over a vector, and removing or…
RAII definitely removes whole classes of errors.
Re: We have a year to fix security everywhere
#289> This probably sounds like nonsense words or hysterical overreacting to most people, so here's what that means: "GLM" is a kind of LLM (AI) [...] The post also sounds like that to people that understand the technology. Calling that out like this and trying to pin that assessment to lack of knowledge is not a get-out-of-jail-free card, nor a good move. __ Edit: Having spent some time letting the article marinate in m…
> What is it? Can it be this autonomous terrifying entity or can it not be? the difference between attack and defense is that attacks can be throwaway code. it's much easier to let an llm hack out a prototype than to get it to build maintainable code that people want to read and review. it's not enough to get Daybreak or Mythos to write you a patch, you need the author of the project to accept and merge it.
Let’s say I’m empowered to patch and deploy. Even then, the HuggingFace hack showed that proven exploits will be automatically disseminated via rogue messaging. Could defenders ever have a system like that?
Re: We have a year to fix security everywhere
#290The standard strategy of a security salesman since 1945. Develop dangerous weapons, show the damage they can do, and sell security cover to the terrified people. Every single piece of technology did this. As a side effect or direct effect, they make bad guys more powerful and then keep on piling up new tech to deal with that. The cycle continues.
Since 1945? Friend, this has been the case since the invention of the pointy stick.
Itself in reaction to someone wielding a big rock