Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Can Now Design Functional Viruses. Should We Worry?

spectrum.ieee.org

Post reply on HN
Print view View original

111–120 of 157 posts

Re: AI Can Now Design Functional Viruses. Should We Worry?

#111

Post by alpineidyll3 »

This happened a full year ago. A) you should immediately discount the expertise of people just learning this now B) a designed pathogen will kill a person this century, full stop, no way to prevent that. The people who invented fission were not so stupid they imagined the bomb wouldn't occur, indeed they assumed it would. I have no idea what became of the wisdom of those people. C) just like nuclear technology that doesn't mean a prohibition is called for. Intelligence is.

What we _can_ do is invest in detection, rapid response technology (vaccines etc), and attempt to mitigate and stratify the possibilities of this happening in the worse ways.

Today's knee jerk prohibitions, and fanciful notions of prohibition working are not facilitating those changes.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#112

Post by alpineidyll3 »

SubiculumCode wrote: post #45

AI is an existential threat, full stop. It can also help us solve many of our greatest problems. Let's not let it solve those problems by removing us as the quickest shortcut.

The sun is an existential threat. It will envelop earth in a few billion years, are you going to shut it off too?

Re: AI Can Now Design Functional Viruses. Should We Worry?

#113

Post by estearum »

smallmancontrov wrote: post #105
estearum wrote: post #95

Earlier quoted context omitted.

Define "reasonable precautions"? Here's a red team exercise where researchers successfully bought fragments of the best known pathogen in the world, never mind a novel one. What reasonable precautions failed to catch this but we could expect to catch a novel virus that "kinda looks like" a known virus? https://www.statnews.com/2024/05/08/shouldnt-be-easy-buy-syn...

So shredding, mutating, and reassembling the genome is the minimum bar to slip it past the scanning? I thought you said you could just order it?

This may have flown past you, but we’re commenting on an article about trivially creating novel pathogens

The shredding, mutating, reassembling is all it took to get the most well known pathogen in the world by the controls that you are suggesting are sufficiently tight

Re: AI Can Now Design Functional Viruses. Should We Worry?

#114

Post by estearum »

smallmancontrov wrote: post #101
estearum wrote: post #92

Earlier quoted context omitted.

The only difference between people who are quite concerned and you is that you are either 1) actually blind to or 2) choosing to ignore even the first derivative on those barriers. The people "got LLM-brain" are saying: "Wow those barriers are coming down fast" And you're saying: "There are still barriers" Both are true, but only one is helpful for thinking about the future. The other is hardly even helpful for think…

So what's your proposal? Lie about the risks enthusiastically enough that someone else does something useful?

Huh? Is "what's your proposal" your reaction to anyone who's alarmed by a problem? Pretty weird if so. Most problems are such that the people who should be alarmed by them is much, much larger than the set of people who could reasonably solve them.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#115

Post by estearum »

timr wrote: post #102
estearum wrote: post #86

Earlier quoted context omitted.

Uhhh right... but all the other hard parts are being solved too, correct? LLMs are clearly going to solve a part that, 10 years ago, one would've assumed would be one of the highest and last barriers to go down. Without LLMs, a reasonable security apparatus would be: Step 1. Disallow dissemination and synthesis of known pathogens Step 2. Watch closely anyone with the decades of training required to come up with their…

> Uhhh right... but all the other hard parts are being solved too, correct? LLMs are clearly going to solve a part that, 10 years ago, one would've assumed would be one of the highest and last barriers to go down. No. No, the barriers I'm talking about aren't coming down. Not quickly anyway. No, nobody assumed this was the "highest and last barrier". That's the point you keep missing. LLMs are not "solving" the rate-…

Why don’t you just scroll up and answer with your timeline estimates so we can talk about something concrete instead of playing this goofy shell game?

“No no some other step is hard. I won’t say what it is. No it’s not getting easier, miraculously, but I can’t say what it is.”

Re: AI Can Now Design Functional Viruses. Should We Worry?

#116

Post by conorcleary »

piloto_ciego wrote: post #93
SubiculumCode wrote: post #45

AI is an existential threat, full stop. It can also help us solve many of our greatest problems. Let's not let it solve those problems by removing us as the quickest shortcut.

I'm a wildly pro-AI dude, but the idea that this is an existential threat makes me think one thing: "you consume too much dystopian content." A lot of things could be an existential threat, nuclear weapons, bio-weapons, climate change, the end of the Flynn effect, cratering birth rates, whatever. You can always pick out a particularly dystopian vision of the future if you're so inclined. The doom narrative is sells a…

'decide'.. look at how pro-ai people surrender to it, and how pro-war ai is...

Re: AI Can Now Design Functional Viruses. Should We Worry?

#118

Post by piloto_ciego »

conorcleary wrote: post #116
piloto_ciego wrote: post #93

Earlier quoted context omitted.

I'm a wildly pro-AI dude, but the idea that this is an existential threat makes me think one thing: "you consume too much dystopian content." A lot of things could be an existential threat, nuclear weapons, bio-weapons, climate change, the end of the Flynn effect, cratering birth rates, whatever. You can always pick out a particularly dystopian vision of the future if you're so inclined. The doom narrative is sells a…

'decide'.. look at how pro-ai people surrender to it, and how pro-war ai is...

AI as pro-war? I do not see that take.

I see pro-war people using AI, but AI being intrinsically pro-war? I beg to differ, if anything, most of the models coming out of the frontier labs seem pretty aligned with anti-war stances. Of course that can change, but yeah.

And "decide" is exactly right. We can decide to not do bad things. You have agency. Stop reading propaganda that is meant to break your spirit and go for a walk, seriously.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#119

Post by salawat »

saulpw wrote: post #104
piloto_ciego wrote: post #96

Earlier quoted context omitted.

Actually, I've seen the take that the previous poster has made too. The wikipedia page is pretty good IMO. Proof of a hypothetical future event is pretty hard to get, but here's the wiki rabbit hole (if you're anything like me you'll find this fascinating and spend the next couple hours clicking links - you're welcome!). https://en.wikipedia.org/wiki/Nuclear_holocaust#Likelihood_o... Annie Jacobsen is awesome, her bo…

It's easier to imagine the end of the world than the end of capitalism.

Only if you're a capitalist. Upton Sinclair applies.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#120

Post by salawat »

energy123 wrote: post #52
acjohnson55 wrote: post #13

Something I've been worried about for years is the proliferation of technologies that give ever smaller numbers of people with smaller amounts resources the ability to impact billions of people, in potentially irreversible ways. Combined with automation, this is a recipe for disaster. We're clearly lacking in the ability to regulate and govern ourselves, and real world market forces (as opposed to the hypothetical fr…

This is what AI safety activist Connor Leahy calls offense-defense asymmetry. Imagine if a nuke could be created with household materials. We got lucky that wasn't the case, but we didn't know that it wasn't the case before studying physics. If we weren't so lucky that physics happened to turn out that way, we already wouldn't exist, because the offense would be so much stronger than defense, we wouldn't stand a chan…

Keep in mind the world is run by hierarchs. Of course the solution will look hierarchical in nature. The problem is fundamentally speaking, the hierarchism is a self perpetuating problem, because we still have to solve who amongst us can be trusted with the top of the hierarchy, which hierarchism states by definition "most of us can't be". It's like encryption, it's excellent at turning every other hard problem into a key management problem.
Post reply on HN