Live data from Hacker News

I'm sorry, you're not going to die from an AI-engineered supervirus

blog.genesmindsmachines.com

51–60 of 122 posts

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#51
post #47

Native americans had like a 95% population mortality rate from infectious diseases in the first generation of European settlement, no? Measles, smallpox, flu, etc. The mortality rate from those diseases was higher than today because the population had no immunity at all, leading to both higher transmission rates and higher mortality rates. This would be true of an engineered virus as well. And it wasn't just the one…

We might even get to relive history the way things are going…

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#52

The magical level of intelligence required is not really a flaw of the argument's deductions, it is its very premise. If you believe that there is a possibility of an AI working on itself, you get the approximate differential equation d/dt f(t) >= C*f(t). In plain English, at least some proportion of development's progress goes back into future development. The (least case growth, assuming equality) solution to this…

> What we really need for such a statement is concrete physical evidence that a virus is simply incapable of having such facilities.

A major focus of the article is about the fundamental tradeoffs in how viruses operate. Was that not enough for you?

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#53

> When people believe that AI could figure this out on its own, without first running extensive and expensive basic science experiments, they have left the realm of science and have entered what can only be described as religion. I strongly agree with this. Too many people have the Arthur C Clarke quote infecting their brains. So much of science is contingent on the physical world and cannot be determined from first…

Is it so hard to think that super intelligence could piece things together that humans can’t?

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#54

The author seems to be arguing that it is biologically impossible or something to have a virus with the same transmission properties as COVID but higher virulence. I don't understand what principle he is invoking to make this claim.

The cope principle

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#56
post #55

Oh my God THANK YOU. I do not have the words how tired I am of hearing tech people and economists talk about biology.

Do you need to be a physicist to realize an asteroid is going to kill you ?

Arguably yes. I would wager no one killed by an asteroid in the history of people killed by asteroids realized what was happening before they died.

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#57

I'm far from anything resembling an expert in biological weapons, but this seems to me puzzlingly overconfident in the opposite direction. > PhD students in 2026 using state-of-the-art AI software are spending months or years trying to design simple peptide binders that inhibit some enzyme or pull down some protein This confuses me, as state-of-the-art AI software is moving quite quickly. Any software that is current…

If you quote the article, then you are implicitly telling others that you have read it. But it's pretty clear you have not?

methinks it is more likely that I read it but missed the point? Would you be able to share what I misunderstood?

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#58
post #55

Oh my God THANK YOU. I do not have the words how tired I am of hearing tech people and economists talk about biology.

Do you need to be a physicist to realize an asteroid is going to kill you ?

Refereeing who is permitted to say what based upon credentials is a popular sport these days.

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#59
post #12

Thank goodness someone wrote this, I was also thinking about writing an article like this. This is such a massively overblown worry that AI companies who are seeking regulatory capture love to roll out whenever they are trying to gin up some FUD. Creating viruses at home is extremely difficult, even if you have a perfect instruction manual and someone sitting behind you telling you what to do. It's full of trial and…

Viruses probably too hard but I think small scale bioterrorism is much easier and doesn't require fancy enzymes. I think this is actually the best argument for regulation of current models. It's even happened pre-AI https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack

Well we're talking about new types of capabilities here, Sarin is a chemical which can be manufactured with a basic understanding of chemistry and searching with google.com. AI will not allow bad actors to do anything they couldn't have done already with google's 10 blue links.

Again the same logic applies though, it takes significant effort and money to create sarin, it requires a lot of rigor. That is the limiting factor, not the knowledge of how to make it.

If we are worried about people making sarin in their garage we can do many things to control the supply of chemicals needed to make it. We don't need to try to restrict knowledge of chemistry.

"regulation of current models" basically implies that open source models will be heavily restricted if not banned outright and guarantees that intelligence will be owned and gatekept by a duopoly.

Re: I'm sorry, you're not going to die from an AI-engineered supervirus

#60
post #12

Thank goodness someone wrote this, I was also thinking about writing an article like this. This is such a massively overblown worry that AI companies who are seeking regulatory capture love to roll out whenever they are trying to gin up some FUD. Creating viruses at home is extremely difficult, even if you have a perfect instruction manual and someone sitting behind you telling you what to do. It's full of trial and…

Viruses probably too hard but I think small scale bioterrorism is much easier and doesn't require fancy enzymes. I think this is actually the best argument for regulation of current models. It's even happened pre-AI https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack

That really proves the point though worst case scenario, releasing a nerve agent into a crowded subway, multiple subways resulted in a grand total of 14 deaths and 50 serious injuries. bum bum bum It's not that that isn't tragic or sad but more people will die or be maimed in a car crash in the US in the next 2 hours than died in the worst case scenario bioterrorism attack.

So if I have to choose between the slim possibility that a crazy person might develop a bio-weapon and the possibility that Anthropic, et. Al convince the government to regulate and control open source models (because I'm sure the government will only do that for good reasons and never things like covering up scandals, squashing dissent or enhancing their own power) and any competitors. I'm probably going to put my chips on a nutcase in a trailer park developing super sars over regulator capture by the government.

Post reply on HN