Live data from Hacker News

AI Can Now Design Functional Viruses. Should We Worry?

spectrum.ieee.org

101–110 of 157 posts

Re: AI Can Now Design Functional Viruses. Should We Worry?

#101
post #84

Earlier quoted context omitted.

I generally agree -- the actual risk is from serious/state actors, and for those, the barriers are not high. I'm just saying that people are focused on the wrong problem because they've got LLM-brain.

The only difference between people who are quite concerned and you is that you are either 1) actually blind to or 2) choosing to ignore even the first derivative on those barriers. The people "got LLM-brain" are saying: "Wow those barriers are coming down fast" And you're saying: "There are still barriers" Both are true, but only one is helpful for thinking about the future. The other is hardly even helpful for think…

So what's your proposal? Lie about the risks enthusiastically enough that someone else does something useful?

Re: AI Can Now Design Functional Viruses. Should We Worry?

#102
post #82

Earlier quoted context omitted.

No. My argument is that many things in the world are hard in a way that computers cannot solve, and the existence of LLMs does not change that. Synthetic biology is one of those things.

Uhhh right... but all the other hard parts are being solved too, correct? LLMs are clearly going to solve a part that, 10 years ago, one would've assumed would be one of the highest and last barriers to go down. Without LLMs, a reasonable security apparatus would be: Step 1. Disallow dissemination and synthesis of known pathogens Step 2. Watch closely anyone with the decades of training required to come up with their…

> Uhhh right... but all the other hard parts are being solved too, correct? LLMs are clearly going to solve a part that, 10 years ago, one would've assumed would be one of the highest and last barriers to go down.

No.

No, the barriers I'm talking about aren't coming down. Not quickly anyway.

No, nobody assumed this was the "highest and last barrier". That's the point you keep missing.

LLMs are not "solving" the rate-limiting step here. It only looks that way if you don't know what's involved.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#103

Earlier quoted context omitted.

It was dishonest then for you to imply that readily accessible commercial labs didn't take reasonable precautions and it's dishonest now , too.

Define "reasonable precautions"? Here's a red team exercise where researchers successfully bought fragments of the best known pathogen in the world, never mind a novel one. What reasonable precautions failed to catch this but we could expect to catch a novel virus that "kinda looks like" a known virus? https://www.statnews.com/2024/05/08/shouldnt-be-easy-buy-syn...

[deleted]

Re: AI Can Now Design Functional Viruses. Should We Worry?

#104
post #91

Earlier quoted context omitted.

Show me that proof please. So far the counter evidence has not been convincing. FWIW I don’t think nuclear war will annihilate us. But it won’t be pleasant.

Actually, I've seen the take that the previous poster has made too. The wikipedia page is pretty good IMO. Proof of a hypothetical future event is pretty hard to get, but here's the wiki rabbit hole (if you're anything like me you'll find this fascinating and spend the next couple hours clicking links - you're welcome!). https://en.wikipedia.org/wiki/Nuclear_holocaust#Likelihood_o... Annie Jacobsen is awesome, her bo…

It's easier to imagine the end of the world than the end of capitalism.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#105

Earlier quoted context omitted.

It was dishonest then for you to imply that readily accessible commercial labs didn't take reasonable precautions and it's dishonest now , too.

Define "reasonable precautions"? Here's a red team exercise where researchers successfully bought fragments of the best known pathogen in the world, never mind a novel one. What reasonable precautions failed to catch this but we could expect to catch a novel virus that "kinda looks like" a known virus? https://www.statnews.com/2024/05/08/shouldnt-be-easy-buy-syn...

So shredding, mutating, and reassembling the genome is the minimum bar to slip it past the scanning? I thought you said you could just order it?

Re: AI Can Now Design Functional Viruses. Should We Worry?

#106
post #19

The regulation at the DNA Printer level works I think, as long as nobody invents a good at home DNA printer, but if they do that there is no need for custom viruses, somebody can take an existing one that is known to be deadly and just print it.

That’s what I’m thinking. If we can restrict EUV technology to the West then why can’t we restrict DNA manufacturing? Not to the West in this case but to capable governments like with Nuclear power.

Sure we can restrict it, just like the only country that has used nukes offensively restricts nukes.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#107
post #97

Earlier quoted context omitted.

“There is another theory which states that this has already happened.” But the next one (or the first one, if that one wasn’t manmade) will probably be much worse.

You get that the one we had was really really bad for a lot of people, right? A friend of mine buried five members of her immediate family. I feel like we are all somehow memoryhole-ing this massive trauma that nothing in this batshit era makes any sense without remembering. Specifically, I wonder if the thirst for a machine god has anything to do with the activation of the Rapture narrative in the collective unconsc…

I know it was, but you also get that it wasn't all that bad for most people, right?

It occupied this weird middle ground where it was bad enough to kill a pretty significant number of people, but not so bad that we're all shell-shocked survivors suppressing the trauma of burying loved ones. And nobody wants to characterize it like that. Half the people want to act like it was nothing, and the other half want to act like it was the worst catastrophe to afflict the world in modern times.

I did get pretty suck for a few weeks, recovered. My wife ended up with strange nerve damage that had her using a cane in her 40s for about six months, but she got back to normal. A friend got hit by Long Covid and plans out how to nap during the work day so he can make it to quitting time. But I don't know anybody who died from it. I am NOT saying nobody died from it, just that it wasn't so widespread that everybody had direct experience of death, like would be the case in something like a smallpox epidemic. The US saw over a million dead, which is a hell of a lot, but in a country of 350 million it's not "everybody comes out with massive trauma" a lot.

Plagues get a lot worse than that. Bodies weren't left to rot because nobody was left alive to bury them. Nobody straggled into the next town because it wiped out the entire population of their home town. A few years later, lots of people say with a straight face that it was no big deal. Nobody who went through the Black Death was saying it was no big deal a few years later.

I'm afraid Covid set us up for the next pandemic to be far worse than it has to be, because it was just mild enough that a substantial number of people are able to dismiss it. And that attitude is going to carry over the next time some novel virus starts spreading around the world. And that one might kill half the population instead of Edit: and I want to emphasize that this is not a memory-hole thing. In December 2020, one third of Americans polled said that Covid was no more of a threat than the seasonal flu (https://www.ipsos.com/sites/default/files/ct/news/documents/...). A lot of other polls tell a similar story: right in the worst months of the pandemic, a large minority of Americans still thought it was no big deal. That's while it was going on, not in retrospect.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#108
post #104

Earlier quoted context omitted.

Actually, I've seen the take that the previous poster has made too. The wikipedia page is pretty good IMO. Proof of a hypothetical future event is pretty hard to get, but here's the wiki rabbit hole (if you're anything like me you'll find this fascinating and spend the next couple hours clicking links - you're welcome!). https://en.wikipedia.org/wiki/Nuclear_holocaust#Likelihood_o... Annie Jacobsen is awesome, her bo…

It's easier to imagine the end of the world than the end of capitalism.

This is what I'm saying constantly to people, I recognize it makes me fantastically obnoxious, but also, I'm gesturing wildly to the things that are happening and yelling, "THINGS DON'T HAVE TO BE THIS WAY."

Then again, I'm also building my cabin in the woods.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#109

I think it's certain that people will try recreating old viruses or making alternatives to them. It will be too easy. Make 5 variants of small pox, 5 variants of sars. Release them at once all around the world to make it hard to fight. It will be too easy for some crazy angry people to do this. I just hope that our defenses and ability to sequence them and respond.

I don’t think it will be easy. Making a virus involves far more than reading some instructions.

I hope I'm wrong.
Post reply on HN