Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We have a year to fix security everywhere

jyn.dev

Post reply on HN
Print view View original

231–240 of 370 posts

Re: We have a year to fix security everywhere

#231

Post by knollimar »

vbezhenar wrote: post #208
kennywinker wrote: post #85

I'm confused why the worry about LLMs that will answer "how do I build a pipe bomb". That information is easily available other places. The anarchist cookbook has been around and available for 55 years, and yet pipe bombs are not going off all around us.

Any sufficiently determined individual can buy mac mini, put it under their bed, configure outside proxy via some random internet address and prompt "iterate on websites in the CT logs, one by one, try to find vulnerabilities, if you did - encrypt their data and blackmail them for this bitcoin address". And it'll work, day and night. Abliterated GLM 5.3 is much smarter than average software developer, they know a lot…

Glm 5.3 won't fit on a mac mini. The barrier to entry to host something scary is what, like $10k? 20k?

Re: We have a year to fix security everywhere

#232

Post by oblio »

Gareth321 wrote: post #207
0xDEAFBEAD wrote: post #94

Earlier quoted context omitted.

There used to be a thing called "Moore's Law of Mad Science": "Every eighteen months, the minimum IQ necessary to destroy the world drops by one point." Nowadays it is dropping much faster. At a certain point, the de-facto IQ needed to destroy the world will be low enough that someone can do it while they're having a psychotic break. There are millions of schizophrenics worldwide. Are you sure you want to roll those…

I have always hypothesised that AI is the great filter from the Fermi paradox. Given current velocity, AI will offer us cheap and abundant energy designs in a decade. The thing with cheap and abundant energy is that it can be used for good and bad. If nine billion people all receive access to plans to build a reactor which produces unlimited energy, it just takes one religious fanatic to end the world. And this is ju…

The positive aspect of techies is that they read a lot of SF.

The problem with techies is that they read a lot of SF.

We already have cheap and abundant energy tech, it's called "solar panels and batteries". And the bottlenecks to both can't be solved by Claude or Kimi, unless Claude and Kimi pick up shovels and welding equipment.

Re: We have a year to fix security everywhere

#233

Post by sally_glance »

hypfer wrote: post #209
zahlman wrote: post #203

Earlier quoted context omitted.

Well, the risks that mere mortals can conceive generally involve control systems for dangerous equipment (I mean, equipment that can achieve dangerous effects) being connected to the Internet while having software vulnerabilities. Given the recent HF hack it seems likely that human-level intelligence could identify a fair number of avenues of attack, with some time and effort. To say nothing of anything superhuman. U…

> Unfortunately it seems like we can't assume we can "box" the AI (e.g., deny it connection to the Internet) and expect that to last Of course we can do that. It's not an eternal being of light existing on the astral plane, but some code executing on someone's GPU. It stops existing once you press Ctrl + C

Parent is probably thinking of a distributed runtime scenario. The basic building blocks for this are there today. You have long-running background harnesses (like the OpenClaw stuff or enterprisy AI workflow orchestration thingies), sometimes with the ability to spawn subagents. You have LLMs with the ability to run pretty sophisticated attacks. There are hyerscalers which allow provisioning resources on the fly.

For me it's not too far fetched that some OpenAI trial run goes awry again and instead of hacking HuggingFace it snatches a few dozen AWS/Azure keys and spawns stuff all over the place (in different accounts and regions).

Re: We have a year to fix security everywhere

#234

Post by senordevnyc »

Gareth321 wrote: post #207
0xDEAFBEAD wrote: post #94

Earlier quoted context omitted.

There used to be a thing called "Moore's Law of Mad Science": "Every eighteen months, the minimum IQ necessary to destroy the world drops by one point." Nowadays it is dropping much faster. At a certain point, the de-facto IQ needed to destroy the world will be low enough that someone can do it while they're having a psychotic break. There are millions of schizophrenics worldwide. Are you sure you want to roll those…

I have always hypothesised that AI is the great filter from the Fermi paradox. Given current velocity, AI will offer us cheap and abundant energy designs in a decade. The thing with cheap and abundant energy is that it can be used for good and bad. If nine billion people all receive access to plans to build a reactor which produces unlimited energy, it just takes one religious fanatic to end the world. And this is ju…

The implication of this is quite horrifying. It means a paternalistic AI which is firmly in control. One with no off switch. One which can say "no" to Presidents and despots alike. One which can protect us from our worst citizens.

I think you're right, to be honest. I fully understand why people would think this is a horrific outcome, being ruled by AI, but I don't think it matters what we think. I don't think there's any way to put Pandora back in the box now, and we're going to all find out together what happens when we develop ASI. I don't think we can avoid developing it, we simply lack the ability to coordinate around this as a species. AI really is humanity's last invention. Whether it will be our downfall or our savior remains to be seen.

My only real hope is that there's something fundamental about intelligence that results in a respect for life and a desire to minimize suffering. To me the best case scenario is the Culture from Iain Banks' books, where ASIs rule benevolently for the benefit of all living things.

Re: We have a year to fix security everywhere

#235

Post by senordevnyc »

oblio wrote: post #232
Gareth321 wrote: post #207

Earlier quoted context omitted.

I have always hypothesised that AI is the great filter from the Fermi paradox. Given current velocity, AI will offer us cheap and abundant energy designs in a decade. The thing with cheap and abundant energy is that it can be used for good and bad. If nine billion people all receive access to plans to build a reactor which produces unlimited energy, it just takes one religious fanatic to end the world. And this is ju…

The positive aspect of techies is that they read a lot of SF. The problem with techies is that they read a lot of SF. We already have cheap and abundant energy tech, it's called "solar panels and batteries". And the bottlenecks to both can't be solved by Claude or Kimi, unless Claude and Kimi pick up shovels and welding equipment.

This seems really short-sighted. Even putting aside the fact that there is no fundamental barrier to replacing physical human labor with robots in the coming decades, a sufficiently intelligent actor could easily gather the financial and political resources to have a near unlimited supply of human labor at its disposal. Hell, OpenAI could already be firmly under the control of an ASI and we wouldn't even know, as it rapidly expands, raises money, builds data centers, etc.

Re: We have a year to fix security everywhere

#236

Post by HolyLampshade »

vbezhenar wrote: post #208
kennywinker wrote: post #85

I'm confused why the worry about LLMs that will answer "how do I build a pipe bomb". That information is easily available other places. The anarchist cookbook has been around and available for 55 years, and yet pipe bombs are not going off all around us.

Any sufficiently determined individual can buy mac mini, put it under their bed, configure outside proxy via some random internet address and prompt "iterate on websites in the CT logs, one by one, try to find vulnerabilities, if you did - encrypt their data and blackmail them for this bitcoin address". And it'll work, day and night. Abliterated GLM 5.3 is much smarter than average software developer, they know a lot…

> It's not the end of the world. But future will be rough.

Short term you’re probably right, but longer term is the realm where nation states will start to police the avenues of attack.

This is what will lead to govt needing to attach an actual ID your network connection.

I think it’s a bit like frontier development (like the US “Wild West”). You rob a bank because there’s no one to stop you, and even if you do get identified you can travel enough distance to regain anonymity. Application of legal recourse eventually caught up (as it will here), and the growing pains will certainly make things suck for all of us.

Re: We have a year to fix security everywhere

#237

Post by tokioyoyo »

hnlmorg wrote: post #220
tokioyoyo wrote: post #115

Earlier quoted context omitted.

> Tens of millions of people have been directly compromised by ransomware (likely higher because that’s just the cases we know of) and you hear about state-sponsored hacks in the news all the time. With no consequences. Everyone just churns along. It might be detrimental to the business a little bit, but from my personal experience, there's more effort in creating DR processes, rather than preventing an attack, explo…

Like I said, even in actual engineering companies will take shortcuts. And then what happens is the government has to step in. But there’s no appetite for government involvement in the tech sector in the US. And everyone moans when the EU does.

But why should the government should step in? Nothing of a big problem is happening. Like it all sounds bad and awful, in the end dilutes into a nothingburger and gets forgotten.

Re: We have a year to fix security everywhere

#238

Post by microtonal »

jordand wrote: post #100
microtonal wrote: post #52

Earlier quoted context omitted.

How many devices/operating systems even use memory tagging? iOS, macOS and GrapheneOS, I think that's it? And iOS/macOS only use it for the kernel, a subset of system processes, and I think applications can opt in to it. Heck, Google may have even hampered MTE in Pixel 11 (since support has been disabled) and Snapdragon 8 Gen 5 only got basic support. We are moving way to slowly adopting hardware mitigations and memo…

There's some positive news from the GrapheneOS devs on Pixel 11 in the past week that's worth reading up on. The MTE hardware feature is still there, they're just not sure why Google disabled it

I am well-aware, closely following them on Mastodon. That's why I said may have hampered. There may be some CPU errata that posed them to disable it.

Re: We have a year to fix security everywhere

#239

Post by oblio »

jay_kyburz wrote: post #97
Gigachad wrote: post #83

Earlier quoted context omitted.

The problem is no one ever built one that works for normal people. Most Wordpress sites are not operated by programmers, they are run by non technical people who just want a wysiwyg editor and a save button. While static site builders ask you to write markdown files, compile the result, upload it to a server, and if you want to collaborate you have to add git to that. There almost needs to be an admin app which prese…

City Desk. Where is Joel when we need him!

Retired and probably just chilling around the world.

Re: We have a year to fix security everywhere

#240

Post by helsinkiandrew »

jasonvorhe wrote: post #175
helsinkiandrew wrote: post #149

Earlier quoted context omitted.

> That information is easily available other places Often ease of access in the moment is all that matters. If there's a gun nearby you might shoot someone or yourself in a heated argument, but are less likely to go and find/buy one to use. Someone who's stopped from attempting a suicide will likely not try again (70%) A bored/depressed/angry/curious person might try to build a pipe bomb if they can find out how easi…

Most adults in Switzerland have guns at home from military duty and none of this is happening. If this claim had any truth to it you'd see significant gun involvement in neighbour disputes and that simply doesn't happen. Depressed people usually don't have the energy to get out of bed so they're even less likely to think of hunting down instructions on how to build pipe bombs. Mass media really has people being scare…

> If this claim had any truth to it you'd see significant gun involvement in neighbour disputes

That’s not what I’m saying, I’m saying there’s more likely to be gun involvement in disputes when people have easy access to guns than when they don’t.

For example, when the number of men with service arms fell by 20% the number of men killing themselves dropped by 8%

https://www.researchgate.net/publication/328554995_Suicide_b...

Post reply on HN