Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We have a year to fix security everywhere

jyn.dev

Post reply on HN
Print view View original

131–140 of 370 posts

Re: We have a year to fix security everywhere

#131

Post by forgotusername6 »

mark_something wrote: post #104
0xDEAFBEAD wrote: post #94

Earlier quoted context omitted.

There used to be a thing called "Moore's Law of Mad Science": "Every eighteen months, the minimum IQ necessary to destroy the world drops by one point." Nowadays it is dropping much faster. At a certain point, the de-facto IQ needed to destroy the world will be low enough that someone can do it while they're having a psychotic break. There are millions of schizophrenics worldwide. Are you sure you want to roll those…

That law is not based on thorough data. Even a person with a sky high IQ can't destroy the world easily. You need access to stuff that is not easy to get. My guess is that developing a new lethal virus or bacteria that is very infectious, is the easiest way, but even that requires a lot of high tech out of reach of most people. Or hacking into systems that control nuclear missiles, but I think these have "air gaps".

The super intelligent AI wouldn't ask you to do anything. It would find the right people and trick them into doing the necessary steps, whatever they are.

Re: We have a year to fix security everywhere

#132

Post by OJFord »

0xDEAFBEAD wrote: post #119
mark_something wrote: post #104

Earlier quoted context omitted.

That law is not based on thorough data. Even a person with a sky high IQ can't destroy the world easily. You need access to stuff that is not easy to get. My guess is that developing a new lethal virus or bacteria that is very infectious, is the easiest way, but even that requires a lot of high tech out of reach of most people. Or hacking into systems that control nuclear missiles, but I think these have "air gaps".

[flagged]

I've been reading it, knowing it was largely debunked/retracted but feeling it was a 'classic' I 'ought' to read; I could only really recommend it if what you want is a Kahneman autobiography.

Re: We have a year to fix security everywhere

#133

Post by hypfer »

stouset wrote: post #124
hypfer wrote: post #105

Earlier quoted context omitted.

If we think this through, I suppose at the end of this (and a bunch of other developments), there will be authoritarianism again. Which _will_ manage the problem, but at what cost.

Again? There is authoritarianism currently .

The authoritarianism will continue until infosec improves.

Re: We have a year to fix security everywhere

#134

Post by daymanstep »

As LLMs make formal verification cheaper (they can generate proofs that can then be automatically checked) many of the verifiable components of software systems, such as compilers and microkernels, will be verified. I suppose the issue is that the critical bugs are rarely in compilers and microkernels, but more often in applications, such as web browsers, which are more difficult to formally verify.

Re: We have a year to fix security everywhere

#135

Post by 0xDEAFBEAD »

hypfer wrote: post #130
0xDEAFBEAD wrote: post #119

Earlier quoted context omitted.

[flagged]

This is a very roundabout way of saying "Anyone not agreeing with me is simply not smart enough". Which might be true, sometimes, but also might not. And especially will not, if the distinction drawn is between blanket statement "worried about AI" and "not worried about AI".

It's not about being smart, it's about accounting for your own ignorance.

>And especially will not, if the distinction drawn is between blanket statement "worried about AI" and "not worried about AI".

I'm just describing the general pattern I see in cognitive tendencies.

If you can think of a way to make the fundamental point about the limitations of our knowledge in a way that's still compelling but less antagonistic, feel free to suggest how I could've rewritten my comment.

Re: We have a year to fix security everywhere

#136

Post by mentalgear »

1 year left for cybersecurity hardening, I thought so as well. The issue is, even if we get it done: in 1 year the models will be so good in social-engineering that they will be able to extract any information they want anyway. Happy to be falsified here, if anyone has evidence-based arguments.

EDIT: by social-engineering I mean for example: recon company structures, gathering and merging people's data from the dark-web, then using it to bribe/pressure/deceive users.

Re: We have a year to fix security everywhere

#137

Post by Planktonne »

forgotusername6 wrote: post #131
mark_something wrote: post #104

Earlier quoted context omitted.

That law is not based on thorough data. Even a person with a sky high IQ can't destroy the world easily. You need access to stuff that is not easy to get. My guess is that developing a new lethal virus or bacteria that is very infectious, is the easiest way, but even that requires a lot of high tech out of reach of most people. Or hacking into systems that control nuclear missiles, but I think these have "air gaps".

The super intelligent AI wouldn't ask you to do anything. It would find the right people and trick them into doing the necessary steps, whatever they are.

This is groundless speculation; if you're going to go that far off the map, then we don't need to worry because a good AI (blue eyes not red) will have learnt to love by then and will save us.

There is currently no reason to believe that such a superintelligence is likely or would have any of the powers people claim.

Re: We have a year to fix security everywhere

#138

Post by grim_io »

littlecranky67 wrote: post #125
kennywinker wrote: post #85

I'm confused why the worry about LLMs that will answer "how do I build a pipe bomb". That information is easily available other places. The anarchist cookbook has been around and available for 55 years, and yet pipe bombs are not going off all around us.

pipe bombs attached to consumer drones will be a big issue. I predict drones will become illegal for the private sector within the following years.

What? If someone wanted to bomb something, they wouldn't be waiting for drones to arrive.

RC cars and planes existed for many decades already.

Re: We have a year to fix security everywhere

#139

Post by 0xDEAFBEAD »

OJFord wrote: post #132
0xDEAFBEAD wrote: post #119

Earlier quoted context omitted.

[flagged]

I've been reading it, knowing it was largely debunked/retracted but feeling it was a 'classic' I 'ought' to read; I could only really recommend it if what you want is a Kahneman autobiography.

I believe the WYSIATI part replicated. Here's an interview I found with Kahneman which touches on the concept:

https://www.apa.org/monitor/2012/02/conclusions

Re: We have a year to fix security everywhere

#140

Post by ErroneousBosh »

sho wrote: post #6

> On September 22, Apple is releasing the M5 Mac Studio with 256 GB of unified memory [..] it will probably [..] enough to write this snippet of code in 3 seconds The author has obviously never ran an LLM on a mac! In 3 seconds, it will have possibly started to think about maybe scheduling a date to contemplate the planning timeline for processing the second token in your prompt.

Also, LLMs never *write* code snippets, they just pirate them from somewhere else.
Post reply on HN