21–30 of 119 posts
"The problems with which the universe can confront any society are, for practical purposes, infinite in number and endless in variety." - Joseph Tainter AI will really aid the universe in generating more problems. Maybe it's not a supervirus, but hell, maybe it is.
"we could be attacked by an infinity of problems, which means that worrying about all of them is statistically unpractical"
I wonder if there is a name to this phenomena. Where being knowledgeable in one thing (such as computer science) means you incorrectly believe you are an expert in another domain (such as virology)
My sci fi story I came up with was,not that a virus is good to kill us,but a rogue AI is going to use a virus or something to hijack our minds and have us do the physical work it wants done like an ant colony.
Thank goodness someone wrote this, I was also thinking about writing an article like this. This is such a massively overblown worry that AI companies who are seeking regulatory capture love to roll out whenever they are trying to gin up some FUD. Creating viruses at home is extremely difficult, even if you have a perfect instruction manual and someone sitting behind you telling you what to do. It's full of trial and…
It's even happened pre-AI https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack
I strongly agree with this. Too many people have the Arthur C Clarke quote infecting their brains.
So much of science is contingent on the physical world and cannot be determined from first principles i.e. by thinking hard about it. (Sorry Plato, you can’t think your way to the moon.) You need increasingly precise measurements, which means increasingly complex tools which means…supply chains. Science requires supply chains. Magic doesn’t.
Just because the author can't think of a way to cause mass death with an engineered virus, doesn't mean an AI won't be able to think of one. (There are several biological threats the author hasn't considered. I'll just leave it at that.)
The author can't think of what a superintelligence will do, because they are not a superintelligence. But that is not really what people are concerned about, so I think the objections here are quite meaningful. If there is a fully general superintelligence - in the current state of alignment - we will all die for some reason. It doesn't really matter what the reason is.
I'm far from anything resembling an expert in biological weapons, but this seems to me puzzlingly overconfident in the opposite direction. > PhD students in 2026 using state-of-the-art AI software are spending months or years trying to design simple peptide binders that inhibit some enzyme or pull down some protein This confuses me, as state-of-the-art AI software is moving quite quickly. Any software that is current…