“I can’t do it so you can’t” is not a valid argument in the age of AI. That’s the point.
I'm sorry, you're not going to die from an AI-engineered supervirus
111–118 of 118 posts
Re: I'm sorry, you're not going to die from an AI-engineered supervirus
#112Obviously, the scenario portrayed in The Economist article is a little far-fetched. I agree that a teenager won’t plan a deadly virus with ChatGPT anytime soon.
But I think the overarching argument here is mostly correct, a lot of really complicated science has been democratized to the point where it is much more accessible. You don’t need a team of brilliant molecular biologists, biochemists, and bioinformaticians to do what you can do with a few agents today. And the pace is increasing. this is the nature of exponential improvement when you approach what Kurzweil popularly call "singularity"
Sure, today, peptide inhibitors and enzyme design are still out of reach for many tasks, I know because that's what I also, as the post authoer, do for a living for the past 15 years. However, only five years ago, protein structure prediction was an open problem, and everyone in our field now regards it as solved.
I’m not worried about a lone wolf taking advantage of AI to design viruses, but I’m concerned that the bar has been lowered to the point where many rogue governments or terrorist organizations can now access tools and skills that were previously available to very few people with a prohibitively high price tag.
The author here looks at the problem with the tunnel vision of a scientist well versed in the field. I know because I’m one as well, but experience has taught me to be more concerned about the unknown unknowns. There could be a team somewhere working on something we haven’t thought about. It doesn’t have to be a population destroyer, either. It could be a tactical infectious agent or something that isn’t meant to kill everyone but ends up being far more dangerous than it initially seems, with implications that are much longer lived than the team anticipated.
I’m not saying we should ban AI or do anything radical, but I do think that with great power comes great responsibility, and we aren’t dedicating enough effort to the responsibility part. There are many benchmarks measuring AI capabilities, but not enough measuring safety. It is becoming increasingly important to develop those.
Re: I'm sorry, you're not going to die from an AI-engineered supervirus
#113Earlier quoted context omitted.
As a biologist whose PhD research was on the genetics of microbial pathogens, I am far more concerned than the author. I think he trots out a lot of straw-man arguments (e.g., teenager making the weapon, "super"virus, etc.). Those scenarios are purposely far-fetched so that he can ridicule them and thereby leave the impression that all the AI/bioweapon concerns are similarly far-fetched. He spends a lot of "tokens" d…
"they can be small molecules that inactivate essential molecular targets in humans" - aka a poison. I mean we deal with the reality of poisons plenty fine right? This isn't x-risk stuff. I was an experimental synbio person for decades, and I did plenty of microbial pathogen work. I've also worked on frontier LLMs for the last decade. All of this stuff is gated by experimental effort, not AI!
> All of this stuff is gated by experimental effort, not AI!
My post said you still needed to run the experiments, but if you think AI can't help to improve bioweapons compared to humans alone, I disagree.
Re: I'm sorry, you're not going to die from an AI-engineered supervirus
#114Earlier quoted context omitted.
Do you need to be a physicist to realize an asteroid is going to kill you ?
Arguably yes. I would wager no one killed by an asteroid in the history of people killed by asteroids realized what was happening before they died.
Not much of an argument as you just proved the counter point. You don't need a specialist for what is now common knowledge.
Re: I'm sorry, you're not going to die from an AI-engineered supervirus
#115Earlier quoted context omitted.
Is it so hard to think that super intelligence could piece things together that humans can’t?
Just because you know something about a thing, doesn't mean you understand the thing. Just because I know the Kabachnik–Fields reaction exists and have studied it, doesn't mean I can do the Kabachnik–Fields reaction. Just because you're smart doesn't mean the world will bend your way. Ask the legions who score high on IQ tests and end up bitter and alone; unable to achieve even a fraction of their goals (or conventio…
There must be some evolutionary kill switch in humans to prevent us from accepting our irrelevance.
Re: I'm sorry, you're not going to die from an AI-engineered supervirus
#116> When people believe that AI could figure this out on its own, without first running extensive and expensive basic science experiments, they have left the realm of science and have entered what can only be described as religion. I strongly agree with this. Too many people have the Arthur C Clarke quote infecting their brains. So much of science is contingent on the physical world and cannot be determined from first…
How about just take an existing design like
>Scientists condemn 'crazy, dangerous' creation of deadly airborne flu virus https://www.theguardian.com/science/2014/jun/11/crazy-danger...
and either produce that or modify it a bit?
Re: I'm sorry, you're not going to die from an AI-engineered supervirus
#117> “Oh man. I used to think Smith was a person with informed opinions, but reading his take on a topic close to my own expertise makes me question this premise.”
Snap. Mike drop. Why continue? lol. Blogs are so snappy. I love a good hot take, and make many myself. But one thing that bugs me are personal insults. We’re living in the most abusive time I can recall. You want to bitch-slap someone do it with a smile, or just tackle their arguments. Otherwise, get bent.
Re: I'm sorry, you're not going to die from an AI-engineered supervirus
#118> When people believe that AI could figure this out on its own, without first running extensive and expensive basic science experiments, they have left the realm of science and have entered what can only be described as religion. I strongly agree with this. Too many people have the Arthur C Clarke quote infecting their brains. So much of science is contingent on the physical world and cannot be determined from first…
I would say ALL of science, not just "much" of it. The difference between math and science is that in science, hard reality is the bottom line. No matter how good your model, how much data you have, what p values you get against simulations - hard reality wins everytime. Thats also why the so called soft "sciences" aren't sciences at all: they are not dealing with hard, objective reality at all.
Math + real world observation -> science