Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Can Now Design Functional Viruses. Should We Worry?

spectrum.ieee.org

Post reply on HN
Print view View original

81–90 of 157 posts

Re: AI Can Now Design Functional Viruses. Should We Worry?

#81

Post by baggy_trough »

mapBasketWand wrote: post #78
dataviz1000 wrote: post #74

Earlier quoted context omitted.

> AI is an existential threat So is nuclear annihilation. Yet, here we are. > "I occasionally think how quickly our differences worldwide would vanish if we were facing an alien threat from outside this world." -- Ronald Reagan at the United Nations [0] We are going to be fine as long as we remember to be nice to each other, starting towards the people who are adjacent to us. [0] https://youtu.be/dYiUI6y1nKg?si=agkJp…

Having recently read both of Annie Jacobsen’s books — _Nuclear War_ and _Biological War_ — I’m much more concerned about a bioweapon than a nuclear one. Her conclusion to the latter book comes out and says it, paraphrasing here: > Nuclear war is terrifying in its destruction, but the chain of command limits the risk of a rogue actor. Whereas bio weapons can be designed and manufactured in relatively low tech environm…

The good news is that bioweapons can’t kill everyone. A handful of percent will remain to repopulate the Earth.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#82

Post by timr »

estearum wrote: post #69
timr wrote: post #59

Earlier quoted context omitted.

No, the sophomoric LLM argument does not apply here. The world is not a big computer. The point is that the rate-limiting step in producing something infective is a couple million dollars in special lab equipment, skilled practitioners, and incredibly tricky and tedious procedures that take time and repetition to master. So yeah, you can try to pay someone who has that expertise to make your top-sekret superbug (good…

Your example of how easy it is to control outputs behind specialized knowledge is a thing (recreational drugs) that exist in mass quantities every place on the planet at consumer-accessible prices?

No. My argument is that many things in the world are hard in a way that computers cannot solve, and the existence of LLMs does not change that.

Synthetic biology is one of those things.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#83

Post by Eddy_Viscosity2 »

bmacho wrote: post #58
Eddy_Viscosity2 wrote: post #21

Earlier quoted context omitted.

Every person's utopia is someone else's dystopia and vice versa. What you've described is clearly some peoples (ie. already billionaires) version of the best possible world. Not so much for the rest of us though...

I'm so surprised that HN is this much anti-government. I'm a doomer myself, I fear the rich, I fear the terrorists, and I think the only one that - can save my life, - is interested in saving my life - i have some control over is the state. Yes, they are currently snatching kids of (illegal) immigrant parents and it is horrible, but who else would protect me against lone nuts in a basement, or robor armies of trillio…

My comment may have been misinterpreted, it was not meant to be anti-government. It is meant to be anti-centralized control though, regardless of the form it takes.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#84

Post by timr »

smallmancontrov wrote: post #77
timr wrote: post #62

Earlier quoted context omitted.

Yes, that's one of the ones I was talking about. That's not a service that makes you a virus, and making one from an mRNA transcript is not trivial.

And one wouldn't make one from mRNA. This guy doesn't know what he's talking about. Still: the difficulty isn't comparable to EUV machines or refining uranium and does not form a comfortable security barrier.

I generally agree -- the actual risk is from serious/state actors, and for those, the barriers are not high.

I'm just saying that people are focused on the wrong problem because they've got LLM-brain.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#86

Post by estearum »

timr wrote: post #82
estearum wrote: post #69

Earlier quoted context omitted.

Your example of how easy it is to control outputs behind specialized knowledge is a thing (recreational drugs) that exist in mass quantities every place on the planet at consumer-accessible prices?

No. My argument is that many things in the world are hard in a way that computers cannot solve, and the existence of LLMs does not change that. Synthetic biology is one of those things.

Uhhh right... but all the other hard parts are being solved too, correct?

LLMs are clearly going to solve a part that, 10 years ago, one would've assumed would be one of the highest and last barriers to go down.

Without LLMs, a reasonable security apparatus would be:

Step 1. Disallow dissemination and synthesis of known pathogens

Step 2. Watch closely anyone with the decades of training required to come up with their own pathogen

Re: AI Can Now Design Functional Viruses. Should We Worry?

#87

Post by smallmancontrov »

estearum wrote: post #72
smallmancontrov wrote: post #60

Earlier quoted context omitted.

The legit synthesis sites have been scanning and authorizing since before the internet was encrypted. You suggested otherwise, demibabs expressed surprise, and was correct to express surprise because your implication was incorrect. The black market exists -- but comes with black market problems.

If that's the understanding you came away with, it was clarified in my immediately following comment. Glad we agree that there are labs that will solve very hard parts of creating a novel virus, either by accident or by negligence.

It was dishonest then for you to imply that readily accessible commercial labs didn't take reasonable precautions and it's dishonest now, too.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#88

Post by falcor84 »

colingauvin wrote: post #73
dgellow wrote: post #35

Earlier quoted context omitted.

That doesn’t invalidate the point, you’re just saying it’s already a risk

No, I'm saying AI has not introduced any fundamentally new capabilities to anyone in this space. Making a virus that grows in a cell is one thing. Making a virus that is an existential risk to humans is altogether another and would require significant new capabilities that do not exist and are not likely to from these training data. (Pre-existing sequences).

Enzymes haven't introduced any fundamentally new capabilities. Any reaction they catalyze could just happen on its own.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#89

Post by fooblaster »

baggy_trough wrote: post #81
mapBasketWand wrote: post #78

Earlier quoted context omitted.

Having recently read both of Annie Jacobsen’s books — _Nuclear War_ and _Biological War_ — I’m much more concerned about a bioweapon than a nuclear one. Her conclusion to the latter book comes out and says it, paraphrasing here: > Nuclear war is terrifying in its destruction, but the chain of command limits the risk of a rogue actor. Whereas bio weapons can be designed and manufactured in relatively low tech environm…

The good news is that bioweapons can’t kill everyone. A handful of percent will remain to repopulate the Earth.

Nukes also cannot kill everyone. Nuclear winter was proven to not be based on solid evidence, the earth remains very habitable after all nukes have gone off.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#90

Post by [deleted] »

smallmancontrov wrote: post #87
estearum wrote: post #72

Earlier quoted context omitted.

If that's the understanding you came away with, it was clarified in my immediately following comment. Glad we agree that there are labs that will solve very hard parts of creating a novel virus, either by accident or by negligence.

It was dishonest then for you to imply that readily accessible commercial labs didn't take reasonable precautions and it's dishonest now , too.

[deleted]
Post reply on HN