Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I'm sorry, you're not going to die from an AI-engineered supervirus

blog.genesmindsmachines.com

Post reply on HN
Print view View original

71–80 of 125 posts

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#71

Post by areoform »

mrits wrote: post #53
janalsncm wrote: post #27

> When people believe that AI could figure this out on its own, without first running extensive and expensive basic science experiments, they have left the realm of science and have entered what can only be described as religion. I strongly agree with this. Too many people have the Arthur C Clarke quote infecting their brains. So much of science is contingent on the physical world and cannot be determined from first…

Is it so hard to think that super intelligence could piece things together that humans can’t?

Just because you know something about a thing, doesn't mean you understand the thing.

Just because I know the Kabachnik–Fields reaction exists and have studied it, doesn't mean I can do the Kabachnik–Fields reaction.

Just because you're smart doesn't mean the world will bend your way. Ask the legions who score high on IQ tests and end up bitter and alone; unable to achieve even a fraction of their goals (or conventional success).

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#72

Post by 01284a7e »

ghurtado wrote: post #21
01284a7e wrote: post #11

"The problems with which the universe can confront any society are, for practical purposes, infinite in number and endless in variety." - Joseph Tainter AI will really aid the universe in generating more problems. Maybe it's not a supervirus, but hell, maybe it is.

That quote is very interesting, because I think I interpreted it the opposite of you: "we could be attacked by an infinity of problems, which means that worrying about all of them is statistically unpractical"

I interpret it similarly, I think - in that you can't totally predict what's going to be the thing that really bites you in the ass.

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#73

Post by georgemcbay »

jsnell wrote: post #47

Native americans had like a 95% population mortality rate from infectious diseases in the first generation of European settlement, no? Measles, smallpox, flu, etc. The mortality rate from those diseases was higher than today because the population had no immunity at all, leading to both higher transmission rates and higher mortality rates. This would be true of an engineered virus as well. And it wasn't just the one…

> So I don't find the bulk of the article about it being impossible for a virus to have this effect all that compelling.

I think the other flaw the article has it that it assumes the bad actor is doing targeted design. That they or the AI has some specific virus in mind meeting a set of predefined criteria and they are fighting the biology to hit that predefined target.

But what if the AI-enabled bad actor is a true chaos agent and has no defined goal and they are using tooling that AI helped them construct to perform genetic reassortment pretty much at random to see what happens?

Most of what they create won't really do anything. But some of it will. And there's a low but non-zero probability of Something Really, Really Bad.

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#74

Post by samplatt »

mrits wrote: post #53
janalsncm wrote: post #27

> When people believe that AI could figure this out on its own, without first running extensive and expensive basic science experiments, they have left the realm of science and have entered what can only be described as religion. I strongly agree with this. Too many people have the Arthur C Clarke quote infecting their brains. So much of science is contingent on the physical world and cannot be determined from first…

Is it so hard to think that super intelligence could piece things together that humans can’t?

That's perfectly believable! Just a pity one hasn't been invented yet, and going by, well... ...everything, never will be.

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#76

Post by jeremyjh »

andai wrote: post #66
jeremyjh wrote: post #14

Earlier quoted context omitted.

The author can't think of what a superintelligence will do, because they are not a superintelligence. But that is not really what people are concerned about, so I think the objections here are quite meaningful. If there is a fully general superintelligence - in the current state of alignment - we will all die for some reason. It doesn't really matter what the reason is.

Well, at that point it doesn't even need to be a virus. It can just be nanobots that atomize your liver. The substrate is a mere technicality.

Right. A superintelligence will kill us in one or more of thousands of possible ways for thousands of possible reasons, perhaps none of which we may understand. Short of a superintelligence - the difficulties of viral bioweapon engineering the author points out are quite meaningful.

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#77

Post by thaumasiotes »

areoform wrote: post #68
janalsncm wrote: post #27

> When people believe that AI could figure this out on its own, without first running extensive and expensive basic science experiments, they have left the realm of science and have entered what can only be described as religion. I strongly agree with this. Too many people have the Arthur C Clarke quote infecting their brains. So much of science is contingent on the physical world and cannot be determined from first…

I want to go further. A lot of people talking about AI bioweapons have no idea just how dangerous these weapons are. Or, rather they think they do, but unfortunately, there's a gulf in understanding between practitioners and commentators. I blame industrial illiteracy and post-literacy, but that's another discussion for another time. Anything that can kill other people can also kill you. I've posted this list before,…

> It's worth repeating again,

> Anything lethal enough to kill other humans is lethal enough to kill you.

It's also completely false; you carry tons of diseases that are capable of killing isolated islanders and incapable of hurting anyone connected to the general run of humanity.

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#78

Post by BatFastard »

Has anyone read https://en.wikipedia.org/wiki/Biological_War:_A_Scenario

Sounds like there already exist some pretty crazy biological weapons in existence. I dont know if it will be 5 years or 20 years, but at some point some fanatic will try to kill the rest of us.

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#79

Post by Scramblejams »

gerdesj wrote: post #10

Ye domesday weapon will not be a disenchanted teenager whipping up COVID-19 variants (whatever that means) and spraying them on people. The real deal needs a lab messing around with viruses and whatever happened a few years ago ... and it did not destroy humanity.

> The real deal needs a lab messing around with viruses and whatever happened a few years ago ... and it did not destroy humanity.

That observation maybe holds if the lab from a few years ago correctly selected the virus for the greatest overall lethality (why not MERS or Ebola or something else?) and made its best effort to turn it into a bioweapon. I don't know about you, but I don't think that's quite what happened...

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#80

Post by akomtu »

janalsncm wrote: post #27

> When people believe that AI could figure this out on its own, without first running extensive and expensive basic science experiments, they have left the realm of science and have entered what can only be described as religion. I strongly agree with this. Too many people have the Arthur C Clarke quote infecting their brains. So much of science is contingent on the physical world and cannot be determined from first…

A morally bankrupt scientist would give AI the hands it needs. Or a military general who thinks he must use AI and any means possible to win, because his enemy may use AI first.
Post reply on HN