Live data from Hacker News

I'm sorry, you're not going to die from an AI-engineered supervirus

blog.genesmindsmachines.com

61–70 of 124 posts

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#61
post #47

Native americans had like a 95% population mortality rate from infectious diseases in the first generation of European settlement, no? Measles, smallpox, flu, etc. The mortality rate from those diseases was higher than today because the population had no immunity at all, leading to both higher transmission rates and higher mortality rates. This would be true of an engineered virus as well. And it wasn't just the one…

This number gets cited a lot but it's actually the excess mortality of post-European contact, and it's not content wide, it's specific to Mexico. It also includes climate change, famine, war, relocation, etc.

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#62

Just because the author can't think of a way to cause mass death with an engineered virus, doesn't mean an AI won't be able to think of one. (There are several biological threats the author hasn't considered. I'll just leave it at that.)

I am not sure what you mean here. There exist plenty high-concern biological threats that dont need any AI help. Human oncovirus design is low on my concern list (immunity is diverse), and in any case it does not need AI—rather labspace. Disgruntled high schoolers or undergrad chemists can do way more damage from readily available materials without AI and without research delay. As can nature (or amateur biologists without AI) by mixing bats with their animal of choice and waiting a while. If/when any scary global events like covid happen again, I sure hope we have true superintelligence to help us navigate it quickly.

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#64
post #55

Earlier quoted context omitted.

Do you need to be a physicist to realize an asteroid is going to kill you ?

Refereeing who is permitted to say what based upon credentials is a popular sport these days.

I would never advocate they not be allowed to say it. But I also am sick of hearing wildly unrealistic takes.

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#65

Just because the author can't think of a way to cause mass death with an engineered virus, doesn't mean an AI won't be able to think of one. (There are several biological threats the author hasn't considered. I'll just leave it at that.)

I was a biophysicist and experimental biologist for decades, I designed viral vectors and cell therapies, etc. What protects us here isn't our ignorance vs an "ASI". It's the fact that the complexity of molecular physics scales exponentially in particle number. No amount of thinking is going to punch through that. You've got to do experiments. The slow loop through reality is not optional, and in the case of "supervi…

We're already doing those experiments. Tailor made mRNA vaccines targeted to one's own specific cancer mutations can be bought right now. The techniques are getting more sophisticated and more targeted every year. Our ability to predict what happens at these levels is improving by leaps and bounds too thanks to AI like AlphaFold. Anyone can be reasonably confident that no AI or teen can do this now or in the next few years, but are you really so confident what might be possible in 10 years?

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#66

Just because the author can't think of a way to cause mass death with an engineered virus, doesn't mean an AI won't be able to think of one. (There are several biological threats the author hasn't considered. I'll just leave it at that.)

The author can't think of what a superintelligence will do, because they are not a superintelligence. But that is not really what people are concerned about, so I think the objections here are quite meaningful. If there is a fully general superintelligence - in the current state of alignment - we will all die for some reason. It doesn't really matter what the reason is.

Well, at that point it doesn't even need to be a virus. It can just be nanobots that atomize your liver. The substrate is a mere technicality.

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#67

Earlier quoted context omitted.

The author can't think of what a superintelligence will do, because they are not a superintelligence. But that is not really what people are concerned about, so I think the objections here are quite meaningful. If there is a fully general superintelligence - in the current state of alignment - we will all die for some reason. It doesn't really matter what the reason is.

please define fully general superintelligence & current state of alignment is this a tautology?

Not the guy but, alignment seems to be several decades away, while self-improvement seems to be several years away.

(Also you don't even need superintelligence to do a lot of damage. In fact you don't need any intelligence!)

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#68

> When people believe that AI could figure this out on its own, without first running extensive and expensive basic science experiments, they have left the realm of science and have entered what can only be described as religion. I strongly agree with this. Too many people have the Arthur C Clarke quote infecting their brains. So much of science is contingent on the physical world and cannot be determined from first…

I want to go further. A lot of people talking about AI bioweapons have no idea just how dangerous these weapons are. Or, rather they think they do, but unfortunately, there's a gulf in understanding between practitioners and commentators.

I blame industrial illiteracy and post-literacy, but that's another discussion for another time.

Anything that can kill other people can also kill you.

I've posted this list before, and I will post this again and again until it sinks in, but this is a real list of real incidents that happened and will continue to happen as long as we study these entities.

    Researcher Nikolai Ustinov was lethally infected with the Marburg virus after accidentally pricking himself with a syringe used for inoculation of guinea pigs. 

    Dora Lush died after accidentally pricking her finger with a needle containing lethal scrub typhus while attempting to develop a vaccine for the disease

    A 23-year-old laboratory assistant at the London School of Hygiene and Tropical Medicine, was infected with smallpox after observing the harvesting of live smallpox virus from eggs without isolation cabinets at that time. The assistant was hospitalised and before being isolated, she infected two visitors to a patient in an adjacent bed, both of whom died. They in turn infected a nurse, who survived

    Ebola laboratory infection by the accidental stick of contaminated needle in the United Kingdom
If you are designing a supervirus that spreads via water, air etc., you better have one hell of a cleanroom, cabinets, manipulation equipment, incinerators (plural), hydrolysis and steam systems, multiple disinfection protocols...

The list is nearly endless, and you can't handwavium it awway.

It doesn't matter if it's "all robots" or not. Even if it's "just robots."

If you're developing something that contagious, then it will start killing animals and it will spread. This has happened multiple times when organizations have screwed up maintenance, see this example from the UK,

    The 2007 United Kingdom foot-and-mouth outbreak was the accidental discharge of virus FMDV BFS 1860 O from a laboratory of the Institute for Animal Health in Pirbright, through possible leakage from broken pipework and via unsealed overflowing manholes, leading to foot-and-mouth disease infections at multiple nearby farms and the culling of over 2,000 animals
It doesn't matter that the robots are out in the middle of nowhere.

If you get sloppy, it will end up in the wastewater run off which will then get picked up and sequenced,

     Routine wastewater surveillance of the vaccine production facility at Utrecht Science Park/Bilthoven [nl] detected infectious poliovirus from a sample collected on 15 November 2022. Full genome sequencing indicated the sample was shedded from an active human infection of wild poliovirus type 3 (WPV3), and further testing of all employees with access to WPV3 found one employee was infected. The employee was isolated until their polio infection was resolved. It remains unclear how the employee became infected given the biosafety measures used at the facility
-

A lot of the people talking about bioweapons of doom are saying that standard graduate level knowledge is an existential risk to humanity.

It's worth repeating again,

Anything lethal enough to kill other humans is lethal enough to kill you.

And if you don't know what you're doing — and for this argument they're talking about people who have to ask a LLM "how do I spanish flu?," the number of ways you will die far outnumber the ways you can succeed.

I am writing a piece on the topic, please contact me if this is your field of expertise.

I am quoting as many primary sources as possible as it's not just some random on the internet, and I've gone back as far as the 1900s to find the many, many, many ways in which these statements are absurd, and misunderstands the potential of their technology.

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#69

Just because the author can't think of a way to cause mass death with an engineered virus, doesn't mean an AI won't be able to think of one. (There are several biological threats the author hasn't considered. I'll just leave it at that.)

I was a biophysicist and experimental biologist for decades, I designed viral vectors and cell therapies, etc. What protects us here isn't our ignorance vs an "ASI". It's the fact that the complexity of molecular physics scales exponentially in particle number. No amount of thinking is going to punch through that. You've got to do experiments. The slow loop through reality is not optional, and in the case of "supervi…

>(And please knock off the bio infohazard act - I routinely get asked about scenarios that AIxBio safety people cook up and they always involve some howler misunderstanding of basic facts in biology or medicine. Just talk about an idea if you have it.)

Yeah, nice try ;)

Post reply on HN