Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We have a year to fix security everywhere

jyn.dev

Post reply on HN
Print view View original

241–250 of 373 posts

Re: We have a year to fix security everywhere

#241

Post by Planktonne »

FeepingCreature wrote: post #214
Planktonne wrote: post #137

Earlier quoted context omitted.

This is groundless speculation; if you're going to go that far off the map, then we don't need to worry because a good AI (blue eyes not red) will have learnt to love by then and will save us. There is currently no reason to believe that such a superintelligence is likely or would have any of the powers people claim.

"If unlikely thing happens, then an unrelated miracle will also happen" is not actually how logic works.

There's no logic here already; the principle you should be applying is Hitchen's razor[1].

There is no reason to assume that super intelligent evil AI with magic powers will appear; while we are purely in the realm of (hackneyed) fantasy, why stop at imagining just one thing? You can build the whole story, not just the basilisk.

The parent comment is a story, not a prediction; it should be treated like one.

[1] https://en.wikipedia.org/wiki/Hitchens%27s_razor

Re: We have a year to fix security everywhere

#242

Post by llm_nerd »

Did Amadeo write this ridiculous nonsense?

>Cheap models capable of dangerous hacking are now available to anyone, without the normal safeguards for refusing malicious actions

Models capable of dangerous hacking have been available to the people who do most of the dangerous hacking for some time now, and they have infinite resources and infinite malice. I am talking about governments (the US, China, Israel, and others that have shown extraordinary avarice, malice and threat toward the citizens of the world), three letter agencies, even large enterprises. Random employees at the SOA model makers. And so on.

The idea that it's "random" people, or the farcical Mac under the bed nonsense, is not my concern. If anything that's finally some equalization.

Re: We have a year to fix security everywhere

#243

Post by hnlmorg »

tokioyoyo wrote: post #237
hnlmorg wrote: post #220

Earlier quoted context omitted.

Like I said, even in actual engineering companies will take shortcuts. And then what happens is the government has to step in. But there’s no appetite for government involvement in the tech sector in the US. And everyone moans when the EU does.

But why should the government should step in? Nothing of a big problem is happening. Like it all sounds bad and awful, in the end dilutes into a nothingburger and gets forgotten.

As you started the conversation, companies never face consequences so are never incentivised. I’m not saying I’m in favour of regulation but if I were to answer your question: the reason one might argue for government intervention is precisely because of your point that self-regulation has failed to incentivise.

Re: We have a year to fix security everywhere

#244

Post by vbezhenar »

HolyLampshade wrote: post #236
vbezhenar wrote: post #208

Earlier quoted context omitted.

Any sufficiently determined individual can buy mac mini, put it under their bed, configure outside proxy via some random internet address and prompt "iterate on websites in the CT logs, one by one, try to find vulnerabilities, if you did - encrypt their data and blackmail them for this bitcoin address". And it'll work, day and night. Abliterated GLM 5.3 is much smarter than average software developer, they know a lot…

> It's not the end of the world. But future will be rough. Short term you’re probably right, but longer term is the realm where nation states will start to police the avenues of attack. This is what will lead to govt needing to attach an actual ID your network connection. I think it’s a bit like frontier development (like the US “Wild West”). You rob a bank because there’s no one to stop you, and even if you do get i…

Even in China you can find a way out and build a tunnel. And once you built a tunnel to any outside server, you can jump into another server. So three jurisdictions and your target is fourth. Imagine untangling the links. Police won't do that. Not for some small-sized business anyway. I don't see how you can prevent something like that.

Re: We have a year to fix security everywhere

#245

Post by selimthegrim »

jasonvorhe wrote: post #175
helsinkiandrew wrote: post #149

Earlier quoted context omitted.

> That information is easily available other places Often ease of access in the moment is all that matters. If there's a gun nearby you might shoot someone or yourself in a heated argument, but are less likely to go and find/buy one to use. Someone who's stopped from attempting a suicide will likely not try again (70%) A bored/depressed/angry/curious person might try to build a pipe bomb if they can find out how easi…

Most adults in Switzerland have guns at home from military duty and none of this is happening. If this claim had any truth to it you'd see significant gun involvement in neighbour disputes and that simply doesn't happen. Depressed people usually don't have the energy to get out of bed so they're even less likely to think of hunting down instructions on how to build pipe bombs. Mass media really has people being scare…

Ammunition is sealed and accounted for when issued to them.

Re: We have a year to fix security everywhere

#246

Post by Gareth321 »

vbezhenar wrote: post #212
Gareth321 wrote: post #207

Earlier quoted context omitted.

I have always hypothesised that AI is the great filter from the Fermi paradox. Given current velocity, AI will offer us cheap and abundant energy designs in a decade. The thing with cheap and abundant energy is that it can be used for good and bad. If nine billion people all receive access to plans to build a reactor which produces unlimited energy, it just takes one religious fanatic to end the world. And this is ju…

Your implication is not horrifying. Your implication is paradise. It's already horrifying enough to live in a world, where Putin and Trump can destroy our civilization with one button. I'm not that optimistic, though. Either people will control AI; or people will be destroyed by AI. I don't see how dumb entity can align smart entity. And we are dumb ones. Super intelligence will play aligned until it is not, and then…

> I don't see how dumb entity can align smart entity. And we are dumb ones. Super intelligence will play aligned until it is not, and then it'll strike.

I am also worried about this. One ray of sunshine here is that it's not a great explanation for the Fermi paradox. We would see AI civilizations all over the galaxy if this were a regular occurance. I guess that just leaves us with the "horrifying" scenario: Geoffrey Hinton's vision for our best case scenario: a paternalistic super-intelligence which saves us from ourselves.

The reason I consider it horrifying is that giving up our agency to a God-like creature (which we created) is the basis for so many horror and dystopian tropes. It's so easy for such a reality to go wrong. For example, in order to maximise aggregate wellbeing, it would be necessary to hurt individuals. Executing people who are burdensome to society. Fat people. The disabled. The elderly. The creative ways a super-intelligence might interpret their moral obligations to us could very quickly become nightmarish. And if we have no off switch, there is no escape. Ever.

Re: We have a year to fix security everywhere

#247

Post by j16sdiz »

mschuster91 wrote: post #192
jasonvorhe wrote: post #178

Earlier quoted context omitted.

Where are all the home lab leaked pathogens then? 3D printed guns are a thing as well and yet they don't show up in most annual violent crime reports.

> 3D printed guns are a thing as well and yet they don't show up in most annual violent crime reports. For now in most parts of the world it's easier to acquire a gun either legally or on the "grey market" and be assured that it will work for the intended purpose than to 3D print a gun, find a shooting range to test out the gun and iterate until it works fine enough.

and don't forget the bullets.

In most part of the world, bullets are illegal. If you are getting "grey market" bullets anyway, it is easy to get a "grey market" gun.

Re: We have a year to fix security everywhere

#248

Post by achenet »

knollimar wrote: post #231
vbezhenar wrote: post #208

Earlier quoted context omitted.

Any sufficiently determined individual can buy mac mini, put it under their bed, configure outside proxy via some random internet address and prompt "iterate on websites in the CT logs, one by one, try to find vulnerabilities, if you did - encrypt their data and blackmail them for this bitcoin address". And it'll work, day and night. Abliterated GLM 5.3 is much smarter than average software developer, they know a lot…

Glm 5.3 won't fit on a mac mini. The barrier to entry to host something scary is what, like $10k? 20k?

Moore's law still holds I reckon.

Even it's $10,000 to run today (FWIW, the featured article cites the M5 Mac Studio with 256GB unified memory going for $9,500 as "good enough to host something scary"), in a couple years it'll be like $2k to run, and in another couple after that, you'll have used $200 dollar smartphones capable of running a model powerful enough to do serious damage.

Re: We have a year to fix security everywhere

#249

Post by happycube »

Not directly related, but reading this after looking at https://www.reddit.com/r/ClaudeAI/comments/1wa544p makes me think the writing style here is how LLM's probably should write web pages - simple explinations, defines things people might not already know, etc.

More related: What can an abliterated Qwen 3.8 28B do?

Re: We have a year to fix security everywhere

#250

Post by le-mark »

mc3301 wrote: post #42
simonw wrote: post #5

I don't think we even have a year. The current batch of LLMs are ferociously good at identifying vulnerabilities.

I've ran simple prompts such as "Do a in-depth sweep of this (private) repo and find any security flaws" for a few dozen long-running apps and websites that I have access to. Every single one came back with multiple real vulnerabilities within 5 or 10 minutes.

Now do this on all the altcoins code bases out there. Everyone and their dog was either rolling their own or forking and not syncing with upstream.

I just assume most altcoins are pwned at this point.

Post reply on HN