Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We have a year to fix security everywhere

jyn.dev

Post reply on HN
Print view View original

371–380 of 382 posts

Re: We have a year to fix security everywhere

#371

Post by hnlmorg »

mike_hearn wrote: post #360
hnlmorg wrote: post #338

Earlier quoted context omitted.

You wouldn’t talk about ransomware like that for precisely the reasons you’d described: it’s a poorly defined open ended problem. You should tackle security in the same way you’d tackle any other kind of engineering initiative in IT. You break the problem down to identifiable tasks that can be easily marked as completed or not required (eg like developers track work in a KANBAN or sprint). So to take your ransomware…

Isn't that just sidestepping the issue? Setting up backups isn't a security task, it's just normal IT which businesses do indeed spend on because there are clear goals and predictable budgets. But just being able to restore data isn't the same thing as not getting ransomware. As you say, you can't define the latter as a goal exactly because it's a security goal, and so will turn into an infinitely long checklist of t…

> Isn't that just sidestepping the issue?

No. It’s addressing the risks of the issue.

> Setting up backups isn't a security task, it's just normal IT which businesses do indeed spend on because there are clear goals and predictable budgets.

All IT security issues are just normal IT.

And the processes I described are how you get clear goals and budgets.

> But just being able to restore data isn't the same thing as not getting ransomware.

The backups are an example. It’s not an exhaustive list of countermeasures.

My point is “not getting ransomware” is a vague and undefined goal like “improve performance”, “add monitoring”, “improve UX”, etc. Any initiative in IT needs to have clearly defined objectives that can be broken down and marked as completed when done. It doesn’t matter if that initiative is software development, UI design or security.

> As you say, you can't define the latter as a goal exactly because it's a security goal, and so will turn into an infinitely long checklist of things you could potentially do with no guarantee of payoff.

Exactly. And that’s why my examples are not sidestepping the issue. They’re just definable subtasks around the risk you’ve identified.

Re: We have a year to fix security everywhere

#372

Post by bigyabai »

bleuarff wrote: post #361
bigyabai wrote: post #336

Earlier quoted context omitted.

It would be much harder to deny it if any of you were capable of describing the risks. The current discussion is a national-scale Handwavium mine.

If these systems allow anyone with an internet access or a few grands to run an open model to obtain the necessary information to e.g. build a bomb, spread an infectious agent, hack water or electrical infrastructures, I think that creates significant threats to the population.

These systems don't enable that. Building a bomb, spreading a bioweapon and hacking water/electrical misconfigurations are all trivially possible without AI. Courtesy to Unibomber, Aum Shinrikyo and the IRGC, respectively, for teaching society this.

In any case, how would AI regulation prevent any of these things? Shouldn't we instead focus on limiting access to bomb/bioweapon precursors and securing vulnerable endpoints, since that is the solution that saves lives regardless of how AI disseminates? Regulating AI is expressly dangerous, if it expands that societal blind spot. It'd be like regulating red mercury.

Re: We have a year to fix security everywhere

#373

Post by zahlman »

ErroneousBosh wrote: post #362
sho wrote: post #359

Earlier quoted context omitted.

Well that's funny, because you were making some pretty blanket statements upstream about how they "never" do this and "always" do that. And yet your experience seems extremely limited and niche, by your own admission. Maybe you should experiment a little more. I think you will quickly learn that your previous impression is wrong. The days of them being merely some sort of jumped up autocomplete are years gone.

Look, I know you believe in the Mystical GPU Sky Fairy, but AI is not in any sense "intelligent". It doesn't think. It's a pattern-matching system like an Eliza bot with a huge corpus to draw from. It is not thinking. It cannot think. It cannot create.

None of that has anything to do with anything either of us said to you, and is also well outside of how people are expected to converse here.

Re: We have a year to fix security everywhere

#374

Post by bamboozled »

FeepingCreature wrote: post #349
bamboozled wrote: post #329

Earlier quoted context omitted.

I guess any virus that is super destructive is probably not something that can spread very far as it kills its hosts beforehand ?

Yep, you'd want to put it on a timer. Optimally trigger it off some natural event, maybe temperature, so it can achieve universal penetration before activating.

Sounds a little bit sci-fi I don't think there is any viruses that exist that are just triggered off temperature ?

Re: We have a year to fix security everywhere

#375

Post by joas_coder »

dbdr wrote: post #10

> Invest in formal verification, fuzzing and property testing, and memory-safe languages. LLMs are good at writing Lean and fuzz tests. I don't care whether you use Go or Rust but for the love of god please don't use C or C++ for new code. How accepted is this thinking in your respective domains?

[flagged]

Re: We have a year to fix security everywhere

#376

Post by littlecranky67 »

grim_io wrote: post #138
littlecranky67 wrote: post #125

Earlier quoted context omitted.

pipe bombs attached to consumer drones will be a big issue. I predict drones will become illegal for the private sector within the following years.

What? If someone wanted to bomb something, they wouldn't be waiting for drones to arrive. RC cars and planes existed for many decades already.

RC cars can't even beat a basic stair or fence, and RC planes are difficult to maneuver as they can't stand still/hover. Try to drop a payload from a flying plane at an exact point on the ground - basically impossible. Even when you do the math, the wind+drag will have a major influence. With a drone you can hit precisely within inches.

Re: We have a year to fix security everywhere

#377

Post by 0xDEAFBEAD »

nextlevelwizard wrote: post #354
0xDEAFBEAD wrote: post #169

Earlier quoted context omitted.

We're talking about open-weight models. It only takes one.

And hardware to run it and coherence and knowledge to set it up. This feels like insane doomerism.

So it stops being insane as soon as... some person or organization offers the ablated open-weight model as a service?

Re: We have a year to fix security everywhere

#378

Post by 0xDEAFBEAD »

someguynamedq wrote: post #356
0xDEAFBEAD wrote: post #94

Earlier quoted context omitted.

There used to be a thing called "Moore's Law of Mad Science": "Every eighteen months, the minimum IQ necessary to destroy the world drops by one point." Nowadays it is dropping much faster. At a certain point, the de-facto IQ needed to destroy the world will be low enough that someone can do it while they're having a psychotic break. There are millions of schizophrenics worldwide. Are you sure you want to roll those…

The minimum IQ necessary to swerve and kill a pedestrian is very low. Yet people don't do it :shrug:. It's almost like people don't want to destroy the world

>Yet people don't do it

I'll bet it has happened at least once.

Re: We have a year to fix security everywhere

#379

Post by 0xDEAFBEAD »

bigyabai wrote: post #334
0xDEAFBEAD wrote: post #183

Earlier quoted context omitted.

Humans are inherently bad at reasoning about rare events. In 2019, many people implicitly reasoned that "since there hasn't been a pandemic in the past 3 years, there won't be one in 2020". Bill Gates was one of the few voices arguing that the world was quite vulnerable to a pandemic. Now he's arguing that the world is quite vulnerable to AI.

> many people implicitly reasoned that "since there hasn't been a pandemic ...pandemics are precedented. There's a world of difference between that and a fully hypothetical catastrophe that you can't even describe without looking and feeling silly. Bill Gates is Epstein's pal, and not a very intelligent person to-boot. I trust his liberal musings about as much as I trust Chomsky's hysterical and discredited views con…

>fully hypothetical catastrophe that you can't even describe without looking and feeling silly.

The HuggingFace hack fell into that category not long ago, yet it happened anyways.

Re: We have a year to fix security everywhere

#380

Post by bleuarff »

bigyabai wrote: post #372
bleuarff wrote: post #361

Earlier quoted context omitted.

If these systems allow anyone with an internet access or a few grands to run an open model to obtain the necessary information to e.g. build a bomb, spread an infectious agent, hack water or electrical infrastructures, I think that creates significant threats to the population.

These systems don't enable that. Building a bomb, spreading a bioweapon and hacking water/electrical misconfigurations are all trivially possible without AI. Courtesy to Unibomber, Aum Shinrikyo and the IRGC, respectively, for teaching society this. In any case, how would AI regulation prevent any of these things? Shouldn't we instead focus on limiting access to bomb/bioweapon precursors and securing vulnerable endpo…

These things are of course possible without the help of AI, but I don't think they're trivial. That requires some specific knowledge, and making it easily obtainable is one less barrier to entry for people interested in it. Such attacks would require both knowledge and access, and I think preventing both is a better strategy than focusing on the physical aspect only.
Post reply on HN