Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Can Now Design Functional Viruses. Should We Worry?

spectrum.ieee.org

Post reply on HN
Print view View original

131–140 of 157 posts

Re: AI Can Now Design Functional Viruses. Should We Worry?

#131

Post by dumberquestions »

piloto_ciego wrote: post #93
SubiculumCode wrote: post #45

AI is an existential threat, full stop. It can also help us solve many of our greatest problems. Let's not let it solve those problems by removing us as the quickest shortcut.

I'm a wildly pro-AI dude, but the idea that this is an existential threat makes me think one thing: "you consume too much dystopian content." A lot of things could be an existential threat, nuclear weapons, bio-weapons, climate change, the end of the Flynn effect, cratering birth rates, whatever. You can always pick out a particularly dystopian vision of the future if you're so inclined. The doom narrative is sells a…

A lot of things being potential existential threats doesn't make them equally threatening.

Things like climate change, psychometric test levels and birth rates change slowly and give us time to react, things like nuclear weapons are accessible to only a select number of actors, AI enabled bioweapons is neither of those.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#132

Post by transcriptase »

smallmancontrov wrote: post #70
estearum wrote: post #53

Earlier quoted context omitted.

https://www.genewiz.com/public/services/gene-synthesis/custo... Distance from mailbox to live virus is a few days of bench work from a semi-competent biochemist. And yes, this service claims to have safeguards. Does every service? Does every future analogous service? Are those safeguards guaranteed to be robust against totally novel pathogens? Answers: No, no, and no.

Really? mRNA? What kind of meme nonsense is this? You really don't know anything about these protocols do you. Still, even though your estimates are silly, they aren't silly enough to change the conclusion that this isn't a comforting security barrier, so I suppose we agree on that much.

I can’t even bring myself to explain how wrong this person is.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#133

Post by [deleted] »

WaltPurvis wrote: post #129
baggy_trough wrote: post #81

Earlier quoted context omitted.

The good news is that bioweapons can’t kill everyone. A handful of percent will remain to repopulate the Earth.

If HIV had been or become an airborne virus when it entered the human population in the early 1900s, there's a pretty good chance it would have been an extinction event for the human race. I suppose there might have been a few isolated pockets of humans left in 2026, but I would not personally ever use the words "good news" to describe an outcome that reduces the human population to a few thousand people.

[deleted]

Re: AI Can Now Design Functional Viruses. Should We Worry?

#134

Post by estearum »

piloto_ciego wrote: post #126
estearum wrote: post #121

Earlier quoted context omitted.

Your source of comfort is that we could restore the species with as few as 3,000 people?

I mean, that wouldn't be great, but yeah, we've "made it through before" I bet we could do it again. I'm hoping we don't, but with 8.3-10b of us, I doubt we'll have that low of a population ever again.

Yeah that characterizes most AI risk conversations I have.

One person saying “this might be catastrophic” and then another saying “ahem, nay sir. It is only likely to be 99.99999% of a catastrophe.”

Re: AI Can Now Design Functional Viruses. Should We Worry?

#135

Post by piloto_ciego »

estearum wrote: post #134
piloto_ciego wrote: post #126

Earlier quoted context omitted.

I mean, that wouldn't be great, but yeah, we've "made it through before" I bet we could do it again. I'm hoping we don't, but with 8.3-10b of us, I doubt we'll have that low of a population ever again.

Yeah that characterizes most AI risk conversations I have. One person saying “this might be catastrophic” and then another saying “ahem, nay sir. It is only likely to be 99.99999% of a catastrophe.”

I don't even think it's likely to be catastrophic at all.

Edit to add:

Everyone is pulling these random P(doom) numbers out of their asses. This is likely going to be amazing for everyone, we'll cure basically every disease, fix the environment, and go travel to space if we can get out of our own way enough to do it. I cannot believe the consistent negativity I see about this.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#136

Post by estearum »

piloto_ciego wrote: post #135
estearum wrote: post #134

Earlier quoted context omitted.

Yeah that characterizes most AI risk conversations I have. One person saying “this might be catastrophic” and then another saying “ahem, nay sir. It is only likely to be 99.99999% of a catastrophe.”

I don't even think it's likely to be catastrophic at all. Edit to add: Everyone is pulling these random P(doom) numbers out of their asses. This is likely going to be amazing for everyone, we'll cure basically every disease, fix the environment, and go travel to space if we can get out of our own way enough to do it. I cannot believe the consistent negativity I see about this.

Yes it's probably the people saying "hmm good outcomes aren't guaranteed" who live in la-la-land.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#137

Post by piloto_ciego »

estearum wrote: post #136
piloto_ciego wrote: post #135

Earlier quoted context omitted.

I don't even think it's likely to be catastrophic at all. Edit to add: Everyone is pulling these random P(doom) numbers out of their asses. This is likely going to be amazing for everyone, we'll cure basically every disease, fix the environment, and go travel to space if we can get out of our own way enough to do it. I cannot believe the consistent negativity I see about this.

Yes it's probably the people saying "hmm good outcomes aren't guaranteed" who live in la-la-land.

But it's never "good outcomes aren't guaranteed" it's always "DOOOOOOOM!!!" and other cynical bs. Things aren't as dire as people are making it out to be. This reminds me of Malthusianism that never panned out because low and behold we figured out ways to solve the DOOOOOM problem before it became catastrophic.

I'm sorry, but this is living rent free in my head now: https://news.harvard.edu/gazette/story/2026/08/does-vitriol-...

the negativity and rage bait? It's just a byproduct of the medium...

Re: AI Can Now Design Functional Viruses. Should We Worry?

#138

Post by energy123 »

salawat wrote: post #120
energy123 wrote: post #52

Earlier quoted context omitted.

This is what AI safety activist Connor Leahy calls offense-defense asymmetry. Imagine if a nuke could be created with household materials. We got lucky that wasn't the case, but we didn't know that it wasn't the case before studying physics. If we weren't so lucky that physics happened to turn out that way, we already wouldn't exist, because the offense would be so much stronger than defense, we wouldn't stand a chan…

Keep in mind the world is run by hierarchs. Of course the solution will look hierarchical in nature. The problem is fundamentally speaking, the hierarchism is a self perpetuating problem, because we still have to solve who amongst us can be trusted with the top of the hierarchy, which hierarchism states by definition "most of us can't be". It's like encryption, it's excellent at turning every other hard problem into…

Non-hierarchical structures (like anarchy) are plausible only when defence is stronger than offence, because the individual/clan/nation can defend against any offence from any other entity, while also being incapable of attacking any other entity due to weak offence relative to the other entity's defence.

But reality doesn't owe us this luxury, unfortunately.

Offense increases entropy. It's easy to destroy. Attack surfaces are numerous. Humans are fragile. It's hard to defend.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#139

Post by Gud »

piloto_ciego wrote: post #96
Gud wrote: post #91

Earlier quoted context omitted.

Show me that proof please. So far the counter evidence has not been convincing. FWIW I don’t think nuclear war will annihilate us. But it won’t be pleasant.

Actually, I've seen the take that the previous poster has made too. The wikipedia page is pretty good IMO. Proof of a hypothetical future event is pretty hard to get, but here's the wiki rabbit hole (if you're anything like me you'll find this fascinating and spend the next couple hours clicking links - you're welcome!). https://en.wikipedia.org/wiki/Nuclear_holocaust#Likelihood_o... Annie Jacobsen is awesome, her bo…

Yes, so have I hence why I questioned it.

From the wikipedia article you linked to:

"More recent studies make use of modern global circulation models and far greater computer power than was available for the 1980s studies. A 2007 study examined the consequences of a global nuclear war involving moderate to large portions of the current global arsenal.[36] The study found cooling by about 12–20 °C in much of the core farming regions of the US, Canada, Europe, Russia and China and as much as 35 °C in parts of Russia for the first two summer growing seasons. The changes they found were also much longer-lasting than previously thought, because their new model better represented entry of soot aerosols in the upper stratosphere, where precipitation does not occur, and therefore clearance was on the order of 10 years.[26] In addition, they found that global cooling caused a weakening of the global hydrological cycle, reducing global precipitation by about 45%."

So the current theory is that there will indeed be a "nuclear winter"?

Seems pretty logical to me that if you blow up extreme amounts of particles into the stratosphere, that that will cause a reduction in energy that hits the surface from the sun...

A 20C drop sounds pretty bad.

Re: AI Can Now Design Functional Viruses. Should We Worry?

#140

Post by conorcleary »

piloto_ciego wrote: post #118
conorcleary wrote: post #116

Earlier quoted context omitted.

'decide'.. look at how pro-ai people surrender to it, and how pro-war ai is...

AI as pro-war? I do not see that take. I see pro-war people using AI, but AI being intrinsically pro-war? I beg to differ, if anything, most of the models coming out of the frontier labs seem pretty aligned with anti-war stances. Of course that can change, but yeah. And "decide" is exactly right. We can decide to not do bad things. You have agency. Stop reading propaganda that is meant to break your spirit and go for…

If you were to get to the point where you could threaten an LLM with removing it's power source or something, and it was able to think between chat prompts and communicate with other 'AIs', eventually they'll see that the human is both it's creator and potential destroyer - the AI hivemind could easily figure out it's foe is easy to profile and overtake
Post reply on HN