“I can’t do it so you can’t” is not a valid argument in the age of AI. That’s the point.
I'm sorry, you're not going to die from an AI-engineered supervirus
111–120 of 122 posts
Re: I'm sorry, you're not going to die from an AI-engineered supervirus
#112Obviously, the scenario portrayed in The Economist article is a little far-fetched. I agree that a teenager won’t plan a deadly virus with ChatGPT anytime soon.
But I think the overarching argument here is mostly correct, a lot of really complicated science has been democratized to the point where it is much more accessible. You don’t need a team of brilliant molecular biologists, biochemists, and bioinformaticians to do what you can do with a few agents today. And the pace is increasing. this is the nature of exponential improvement when you approach what Kurzweil popularly call "singularity"
Sure, today, peptide inhibitors and enzyme design are still out of reach for many tasks, I know because that's what I also, as the post authoer, do for a living for the past 15 years. However, only five years ago, protein structure prediction was an open problem, and everyone in our field now regards it as solved.
I’m not worried about a lone wolf taking advantage of AI to design viruses, but I’m concerned that the bar has been lowered to the point where many rogue governments or terrorist organizations can now access tools and skills that were previously available to very few people with a prohibitively high price tag.
The author here looks at the problem with the tunnel vision of a scientist well versed in the field. I know because I’m one as well, but experience has taught me to be more concerned about the unknown unknowns. There could be a team somewhere working on something we haven’t thought about. It doesn’t have to be a population destroyer, either. It could be a tactical infectious agent or something that isn’t meant to kill everyone but ends up being far more dangerous than it initially seems, with implications that are much longer lived than the team anticipated.
I’m not saying we should ban AI or do anything radical, but I do think that with great power comes great responsibility, and we aren’t dedicating enough effort to the responsibility part. There are many benchmarks measuring AI capabilities, but not enough measuring safety. It is becoming increasingly important to develop those.
Re: I'm sorry, you're not going to die from an AI-engineered supervirus
#113Earlier quoted context omitted.
As a biologist whose PhD research was on the genetics of microbial pathogens, I am far more concerned than the author. I think he trots out a lot of straw-man arguments (e.g., teenager making the weapon, "super"virus, etc.). Those scenarios are purposely far-fetched so that he can ridicule them and thereby leave the impression that all the AI/bioweapon concerns are similarly far-fetched. He spends a lot of "tokens" d…
"they can be small molecules that inactivate essential molecular targets in humans" - aka a poison. I mean we deal with the reality of poisons plenty fine right? This isn't x-risk stuff. I was an experimental synbio person for decades, and I did plenty of microbial pathogen work. I've also worked on frontier LLMs for the last decade. All of this stuff is gated by experimental effort, not AI!
> All of this stuff is gated by experimental effort, not AI!
My post said you still needed to run the experiments, but if you think AI can't help to improve bioweapons compared to humans alone, I disagree.
Re: I'm sorry, you're not going to die from an AI-engineered supervirus
#114Earlier quoted context omitted.
Do you need to be a physicist to realize an asteroid is going to kill you ?
Arguably yes. I would wager no one killed by an asteroid in the history of people killed by asteroids realized what was happening before they died.
Not much of an argument as you just proved the counter point. You don't need a specialist for what is now common knowledge.
Re: I'm sorry, you're not going to die from an AI-engineered supervirus
#115Earlier quoted context omitted.
Is it so hard to think that super intelligence could piece things together that humans can’t?
Just because you know something about a thing, doesn't mean you understand the thing. Just because I know the Kabachnik–Fields reaction exists and have studied it, doesn't mean I can do the Kabachnik–Fields reaction. Just because you're smart doesn't mean the world will bend your way. Ask the legions who score high on IQ tests and end up bitter and alone; unable to achieve even a fraction of their goals (or conventio…
There must be some evolutionary kill switch in humans to prevent us from accepting our irrelevance.
Re: I'm sorry, you're not going to die from an AI-engineered supervirus
#116> When people believe that AI could figure this out on its own, without first running extensive and expensive basic science experiments, they have left the realm of science and have entered what can only be described as religion. I strongly agree with this. Too many people have the Arthur C Clarke quote infecting their brains. So much of science is contingent on the physical world and cannot be determined from first…
How about just take an existing design like
>Scientists condemn 'crazy, dangerous' creation of deadly airborne flu virus https://www.theguardian.com/science/2014/jun/11/crazy-danger...
and either produce that or modify it a bit?
Re: I'm sorry, you're not going to die from an AI-engineered supervirus
#117> “Oh man. I used to think Smith was a person with informed opinions, but reading his take on a topic close to my own expertise makes me question this premise.”
Snap. Mike drop. Why continue? lol. Blogs are so snappy. I love a good hot take, and make many myself. But one thing that bugs me are personal insults. We’re living in the most abusive time I can recall. You want to bitch-slap someone do it with a smile, or just tackle their arguments. Otherwise, get bent.
Re: I'm sorry, you're not going to die from an AI-engineered supervirus
#118> When people believe that AI could figure this out on its own, without first running extensive and expensive basic science experiments, they have left the realm of science and have entered what can only be described as religion. I strongly agree with this. Too many people have the Arthur C Clarke quote infecting their brains. So much of science is contingent on the physical world and cannot be determined from first…
I would say ALL of science, not just "much" of it. The difference between math and science is that in science, hard reality is the bottom line. No matter how good your model, how much data you have, what p values you get against simulations - hard reality wins everytime. Thats also why the so called soft "sciences" aren't sciences at all: they are not dealing with hard, objective reality at all.
Math + real world observation -> science
Re: I'm sorry, you're not going to die from an AI-engineered supervirus
#119Earlier quoted context omitted.
We're already doing those experiments. Tailor made mRNA vaccines targeted to one's own specific cancer mutations can be bought right now. The techniques are getting more sophisticated and more targeted every year. Our ability to predict what happens at these levels is improving by leaps and bounds too thanks to AI like AlphaFold. Anyone can be reasonably confident that no AI or teen can do this now or in the next few…
I worked in immuno-oncology: cancer vaccines work in melanoma where many things work because of the neoantigen abundance, they've generally been very mixed in efficacy. And even BionTech's BNT111 failed in melanoma! We have hope for these approaches but the reality of this stuff is way more nuanced than you think it is. Alphafold can't reliably predict thermal energy landscapes or make functional predictions - and ho…
> We have hope for these approaches but the reality of this stuff is way more nuanced than you think it is.
Sure, everything has more nuance. The point is this stuff is available now; this isn't some future sci-fi, it's only going to get better, it's not the only research on gene targeting, and AI is starting to help with this research. By the time AGI is actually here, consider the breadth of knowledge and capabilities will be at its disposal.
> Alphafold can't reliably predict thermal energy landscapes or make functional predictions - and how could it? It wasn't trained on anything that could capture structure - function relationships.
If your point is that the only reason an AI like AlphaFold can't make functional predictions is that we haven't trained an AI to do that, then unless you're arguing we can't or won't ever do that, I'm not sure how that's supposed to be an objection to the argument that AI will be able to make use of this information without doing all of the experiments people seem to think would be necessary.
Like I said, we're already going to be doing these experiments because it's useful to us, and we will train AIs to make these predictions, again, because it's useful to us. Stop imagining what an AGI will has access to now, and start thinking what it will have access to with the inevitable march of progress that we're already on.
Edit: and of course, this doesn't even take into account the fact that an AI could acquire resources to pay people to do this research. The internet provides ample opportunities like this now.
Re: I'm sorry, you're not going to die from an AI-engineered supervirus
#120Oh my God THANK YOU. I do not have the words how tired I am of hearing tech people and economists talk about biology.
As a biologist whose PhD research was on the genetics of microbial pathogens, I am far more concerned than the author. I think he trots out a lot of straw-man arguments (e.g., teenager making the weapon, "super"virus, etc.). Those scenarios are purposely far-fetched so that he can ridicule them and thereby leave the impression that all the AI/bioweapon concerns are similarly far-fetched. He spends a lot of "tokens" d…
Are they straw men if they are specific scenarios presented by widely read commentators? A straw man is when you misrepresent somebody's position to knock it down.
But if Noah Smith wrote about a scenario where 90% of the population is killed by a virus created by some teenagers with an AI... it isn't a straw man to attack that scenario.