Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Can Now Design Functional Viruses. Should We Worry?

spectrum.ieee.org

Post reply on HN
Print view View original

121–130 of 157 posts

Re: AI Can Now Design Functional Viruses. Should We Worry?

#121

Post by estearum »

piloto_ciego wrote: post #93
SubiculumCode wrote: post #45

AI is an existential threat, full stop. It can also help us solve many of our greatest problems. Let's not let it solve those problems by removing us as the quickest shortcut.

I'm a wildly pro-AI dude, but the idea that this is an existential threat makes me think one thing: "you consume too much dystopian content." A lot of things could be an existential threat, nuclear weapons, bio-weapons, climate change, the end of the Flynn effect, cratering birth rates, whatever. You can always pick out a particularly dystopian vision of the future if you're so inclined. The doom narrative is sells a…

Your source of comfort is that we could restore the species with as few as 3,000 people?

Re: AI Can Now Design Functional Viruses. Should We Worry?

#122

Post by estearum »

alpineidyll3 wrote: post #112
SubiculumCode wrote: post #45

AI is an existential threat, full stop. It can also help us solve many of our greatest problems. Let's not let it solve those problems by removing us as the quickest shortcut.

The sun is an existential threat. It will envelop earth in a few billion years, are you going to shut it off too?

If we could, it would be wise to prevent the sun from enveloping the earth at some point before that happened, yes.

(obviously "we" won't be around or whatever, but the point remains: yes, you should solve as many existential threats as you can before they're realized)

Re: AI Can Now Design Functional Viruses. Should We Worry?

#123

Post by estearum »

baggy_trough wrote: post #81
mapBasketWand wrote: post #78

Earlier quoted context omitted.

Having recently read both of Annie Jacobsen’s books — _Nuclear War_ and _Biological War_ — I’m much more concerned about a bioweapon than a nuclear one. Her conclusion to the latter book comes out and says it, paraphrasing here: > Nuclear war is terrifying in its destruction, but the chain of command limits the risk of a rogue actor. Whereas bio weapons can be designed and manufactured in relatively low tech environm…

The good news is that bioweapons can’t kill everyone. A handful of percent will remain to repopulate the Earth.

Not guaranteed whatsoever.

People keep mixing up the fact that it's very hard for a virus to naturally evolve that's both extremely lethal and extremely contagious with the totally distinct claim that it's hard or impossible to create a virus that's both extremely lethal and extremely contagious.

The former is true for obvious reasons. The latter claim is not true at all. Nothing prevents the existence of a virus that's both of those things.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#124

Post by estearum »

mapBasketWand wrote: post #78
dataviz1000 wrote: post #74

Earlier quoted context omitted.

> AI is an existential threat So is nuclear annihilation. Yet, here we are. > "I occasionally think how quickly our differences worldwide would vanish if we were facing an alien threat from outside this world." -- Ronald Reagan at the United Nations [0] We are going to be fine as long as we remember to be nice to each other, starting towards the people who are adjacent to us. [0] https://youtu.be/dYiUI6y1nKg?si=agkJp…

Having recently read both of Annie Jacobsen’s books — _Nuclear War_ and _Biological War_ — I’m much more concerned about a bioweapon than a nuclear one. Her conclusion to the latter book comes out and says it, paraphrasing here: > Nuclear war is terrifying in its destruction, but the chain of command limits the risk of a rogue actor. Whereas bio weapons can be designed and manufactured in relatively low tech environm…

Not sure how much Annie Jacobsen goes into the exact dynamics in Nuclear War, but it may be worth reading Ellsbergs' Doomsday Machine before you find comfort in the "chain of command."

Re: AI Can Now Design Functional Viruses. Should We Worry?

#125

Post by estearum »

rich_sasha wrote: post #85

Don’t we already have plenty of deadly viruses around? Why would you design a virus when rabies, anthrax and Ebola, for starters, exist in the wild?

Naturally evolved viruses tend to have a tradeoff between contagiousness and lethality (just because both features have to evolve somewhat simultaneously). Synthetic viruses have no such difficulty.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#126

Post by piloto_ciego »

estearum wrote: post #121
piloto_ciego wrote: post #93

Earlier quoted context omitted.

I'm a wildly pro-AI dude, but the idea that this is an existential threat makes me think one thing: "you consume too much dystopian content." A lot of things could be an existential threat, nuclear weapons, bio-weapons, climate change, the end of the Flynn effect, cratering birth rates, whatever. You can always pick out a particularly dystopian vision of the future if you're so inclined. The doom narrative is sells a…

Your source of comfort is that we could restore the species with as few as 3,000 people?

I mean, that wouldn't be great, but yeah, we've "made it through before" I bet we could do it again. I'm hoping we don't, but with 8.3-10b of us, I doubt we'll have that low of a population ever again.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#127

Post by haxiomic »

timr wrote: post #67
haxiomic wrote: post #10

Earlier quoted context omitted.

There’s YouTubers who do this [0]. I’d argue estimate the risks of a synthetically induced pandemic of massive impact this century are very high - [0] https://youtu.be/aoczYXJeMY4?is=OqObeGzZB6AKL0Jk

Not even close to the same thing. That's taking an existing viral transfection system and swapping in a single gene. Thermo Fisher sells a kit to do this [1]. It's something you might do a few thousand times on the way to actually generating a novel virus. > I’d argue estimate the risks of a synthetically induced pandemic of massive impact this century are very high I'd argue that it probably already happened, and ha…

A single gene is all that’s needed for human transmission, but multiple genes is not a major step up. This is exactly what I’d expect the most dangerous viruses to be, small gain of function tweaks to existing dangerous viruses, of which there are many to pick from

Re: AI Can Now Design Functional Viruses. Should We Worry?

#128

Post by hoppp »

dotancohen wrote: post #57
hoppp wrote: post #19

The regulation at the DNA Printer level works I think, as long as nobody invents a good at home DNA printer, but if they do that there is no need for custom viruses, somebody can take an existing one that is known to be deadly and just print it.

Aren't technological constraints such as these short lived? Even if s home DNA printer won't be available this decade, the technology will come at some point. It's far better to have both legal and social policy in place before then - though the legal policy will probably not matter at all.

Yes, the tech should arrive eventually but its needs a breakthrough to use readily available chemicals that are not regulated.

You are right, social and legal policy is important but it's not really gonna stop a terrorist organization in Africa.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#129

Post by WaltPurvis »

baggy_trough wrote: post #81
mapBasketWand wrote: post #78

Earlier quoted context omitted.

Having recently read both of Annie Jacobsen’s books — _Nuclear War_ and _Biological War_ — I’m much more concerned about a bioweapon than a nuclear one. Her conclusion to the latter book comes out and says it, paraphrasing here: > Nuclear war is terrifying in its destruction, but the chain of command limits the risk of a rogue actor. Whereas bio weapons can be designed and manufactured in relatively low tech environm…

The good news is that bioweapons can’t kill everyone. A handful of percent will remain to repopulate the Earth.

If HIV had been or become an airborne virus when it entered the human population in the early 1900s, there's a pretty good chance it would have been an extinction event for the human race. I suppose there might have been a few isolated pockets of humans left in 2026, but I would not personally ever use the words "good news" to describe an outcome that reduces the human population to a few thousand people.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#130

Post by [deleted] »

estearum wrote: post #115
timr wrote: post #102

Earlier quoted context omitted.

> Uhhh right... but all the other hard parts are being solved too, correct? LLMs are clearly going to solve a part that, 10 years ago, one would've assumed would be one of the highest and last barriers to go down. No. No, the barriers I'm talking about aren't coming down. Not quickly anyway. No, nobody assumed this was the "highest and last barrier". That's the point you keep missing. LLMs are not "solving" the rate-…

Why don’t you just scroll up and answer with your timeline estimates so we can talk about something concrete instead of playing this goofy shell game? “No no some other step is hard. I won’t say what it is. No it’s not getting easier, miraculously, but I can’t say what it is.”

[deleted]
Post reply on HN