Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I'm sorry, you're not going to die from an AI-engineered supervirus

blog.genesmindsmachines.com

Post reply on HN
Print view View original

41–50 of 123 posts

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#41

Post by alevskaya »

Metacelsus wrote: post #3

Just because the author can't think of a way to cause mass death with an engineered virus, doesn't mean an AI won't be able to think of one. (There are several biological threats the author hasn't considered. I'll just leave it at that.)

I was a biophysicist and experimental biologist for decades, I designed viral vectors and cell therapies, etc. What protects us here isn't our ignorance vs an "ASI".

It's the fact that the complexity of molecular physics scales exponentially in particle number. No amount of thinking is going to punch through that. You've got to do experiments. The slow loop through reality is not optional, and in the case of "superviruses" would require a lot of iterative experimental development in humans.

(And please knock off the bio infohazard act - I routinely get asked about scenarios that AIxBio safety people cook up and they always involve some howler misunderstanding of basic facts in biology or medicine. Just talk about an idea if you have it.)

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#42

Post by hex4def6 »

Now here’s a real risk involving measles that I sometimes worry about. What if the measles virus experienced a set of immune escape mutations where suddenly existing vaccines no longer work? If this happened the consequences would be horrifying. We’d have an immediate, world-wide measles epidemic long before we could develop and roll out a new vaccine. What’s the likelihood this will happen? Informed estimates are that it is low. But realistically speaking, even if the likelihood is low, it’s probably higher than the likelihood that a disgruntled teenager will design a supervirus with AI. So, something to think about.

I'm confused. Would a bit of gain-of-function exploration + measles virus genome = supervirus?

I'm no biologist, but it seems like the genome is readily available. https://www.ncbi.nlm.nih.gov/nuccore/NC_001498.1

And it seems like the spots that have to be mutated are already known about: https://newsnetwork.mayoclinic.org/discussion/researchers-cl...

And this experiment has already been done: https://pmc.ncbi.nlm.nih.gov/articles/PMC8080110/

Five minutes of research suggests the sites -- Loop Epitope etc -- only need a few pairs of mutations each. this feels doable... It's not creating a supervirus from whole cloth, but why bother. Measles without a vaccine would be a catastrophe.

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#44

Post by alevskaya »

skew-aberration wrote: post #24
Dig1t wrote: post #12

Thank goodness someone wrote this, I was also thinking about writing an article like this. This is such a massively overblown worry that AI companies who are seeking regulatory capture love to roll out whenever they are trying to gin up some FUD. Creating viruses at home is extremely difficult, even if you have a perfect instruction manual and someone sitting behind you telling you what to do. It's full of trial and…

Viruses probably too hard but I think small scale bioterrorism is much easier and doesn't require fancy enzymes. I think this is actually the best argument for regulation of current models. It's even happened pre-AI https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack

You can read up on the synthesis of sarin on wikipedia.

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#45

Post by naasking »

janalsncm wrote: post #27

> When people believe that AI could figure this out on its own, without first running extensive and expensive basic science experiments, they have left the realm of science and have entered what can only be described as religion. I strongly agree with this. Too many people have the Arthur C Clarke quote infecting their brains. So much of science is contingent on the physical world and cannot be determined from first…

We're doing that science right now. We're using AI to do it. By the time AGI truly spreads, we'll have a lot of the knowledge you say will be needed. This is simply not as far fetched as you seem to think. Next few years, unlikely. 10-15 years? Increasingly possible. 20+ years, I'd hazard even likely.

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#46

Post by fergal_reid »

I am not a biology person, but, candidly, this is a bad article, because the author is generalizing from the properties of viruses that tend to arise naturally, and using this to argue about the properties of deliberately designed viruses.

Firstly, it's not even clear that naturally occuring viruses can't be extremely lethal. There are examples (eg black death).

Even the existence of rare examples should be concerning, if attempting to argue against the possibility.

But even if there were no natural examples, it could just be because the ecosystem is in a balance, which them unlikely. That doesn't preclude someone from designing one.

To use an analogy. Natural predator prey ecosystems tend to be in balance. You could look around the world, before modern humans, and conclude that superpredators aren't a real threat, because predators that eat their prey too rapidly typically then die out. Thus most ecosystems you observe are in some balanced state.

But this doesn't mean modern humans can't come along, using their superior intelligence, and act as a superpredator. Ask the wolf population of western Europe.

If something is stopping us from doing this more often, it's alignment rather than capability.

You can't generalized from the typical behaviour that you see in a balanced natural ecosystem, and use that to say it proves there are no much more dangerous possibilities.

If the author made first principles argument about the inherent difficulty of doing something bad, even with abundant intelligence, I'd be much more reassured; or about defense offense asymmetry favouring defence.

What's here seems very naive.

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#47

Post by jsnell »

Native americans had like a 95% population mortality rate from infectious diseases in the first generation of European settlement, no? Measles, smallpox, flu, etc.

The mortality rate from those diseases was higher than today because the population had no immunity at all, leading to both higher transmission rates and higher mortality rates. This would be true of an engineered virus as well.

And it wasn't just the one virus, it was an entire continent's infectious disease arsenal landing in overlapping waves. Again, something you'd do if you were trying to maximize damage.

So I don't find the bulk of the article about it being impossible for a virus to have this effect all that compelling.

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#49

Post by colingauvin »

skew-aberration wrote: post #24
Dig1t wrote: post #12

Thank goodness someone wrote this, I was also thinking about writing an article like this. This is such a massively overblown worry that AI companies who are seeking regulatory capture love to roll out whenever they are trying to gin up some FUD. Creating viruses at home is extremely difficult, even if you have a perfect instruction manual and someone sitting behind you telling you what to do. It's full of trial and…

Viruses probably too hard but I think small scale bioterrorism is much easier and doesn't require fancy enzymes. I think this is actually the best argument for regulation of current models. It's even happened pre-AI https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack

Most people do not understand how easy it would be to make anthrax. Technology is not the reason it basically is never done or deployed. AI cannot meaningfully move this needle.

Let's look at it in an inverse way. Making a single drug that impacts a single protein, intelligently, from the ground up, has arguably never actually been done. Despite unbelievable financial incentive. And with AI.

Post reply on HN