My personal worst case scenarios include dying in the next 25 or 30 years as a normal consequence of human old age. Perhaps accelerated scientific development from advanced AIs is my only hope of living longer, at this point.
Yeah but there’s a lot of young Children who aren’t in such a dire position who deserve at least for the chance to live as long as you have?
They will encounter the same problem, eventually. 8 billion humans heading towards a certain death, I think that counts as a catastrophe.
Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…
> The only prior example of world coordination at this level would be nuclear disarmament achieved via the logic of mutually assured destruction
Or eradication of infectious diseases such as polio
I guess it's interesting because the "intelligence" we're inventing is ignorant, incapable of explanation, and likely filled with nasty biases. It's even the more dangerous because of that, mainly because people are viewing it as a source of truth.
If intelligence is all you need to dominate the world, why do some of the most powerful world leaders seem to not be more than a standard deviation above average intelligence (or at least they were before they became geriatric)? Add in the fact that GPT-4 does not "think" very fast, has no arms or legs, needs to run on expensive specialized hardware, and essentially seems like a search algorithm that lacks creativity…
Hey GPT-5, write the code implementing a bioinformatics workflow to design a novel viral RNA sequence to maximize the extermination of human life. The virus genome should be optimized for R-naught and mortality. Perform a literature search to determine the most effective human cellular targets to run the pipeline on. Use off the shelf publicly available state-of-the-art sequence to structure models and protein free-e…
With the current ChatGPT it's already hard to let it insult people. I'm sure safeguards would be built in to prevent this.
Can you potentially circumvent these? Probably, but then again it won't be available for every dimwit, but only people smart enough to know how.
And add in just one peer-level war where one side has their back against the wall. Then give it 100 years where anyone can create such a model on their phone. We’d need a constantly evolving inoculation function to compete. And it would probably lose because the other side has fewer restrictions. In my darker thoughts about this, this is why we see no aliens. To get this to work we need a far smarter entity with no p…
> In my darker thoughts about this, this is why we see no aliens. If AI would be a common great filter we'd expect at least one of them to expand outwards after being the filter?
According to the grabby aliens hypothesis [1] there are reasons to believe that a) humans are early, so nothing else has yet had time to convert the local group into something incompatible with the rise of new technological civilizations, and b) expanding aliens, whether monomaniacal AIs or something else, likely expand close to the speed of light, so we don't get much advance warning before they arrive. However, even if we become grabby ourselves, it could take tens or hundreds of millions of years before our expansion wavefront meets that of another civilization.
Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…
> The only prior example of world coordination at this level would be nuclear disarmament achieved via the logic of mutually assured destruction Or eradication of infectious diseases such as polio
I don't think that counts, because nobody wants to keep polio around, nor does eliminating your own country's polio disadvantage you in any way. Also, it's not as if the elected politicians of each country got together and negotiated an effort to end polio- it was more a concerted effort by WHO, UNICEF, CDC, et al. They just did it under the (correct) assumption that the political entities of the world weren't going to try and stop them.
Two-party iterated prisoner's dilemma is hard enough. Sensible players will coordinate with something like tit-for-tat, but that only works when both parties start off on the right foot. Regardless of initial strategy, the chances of degenerating towards the mutual-defection Nash equilibrium increase with the number of parties. The only prior example of world coordination at this level would be nuclear disarmament ac…
> The only prior example of world coordination at this level would be nuclear disarmament achieved via the logic of mutually assured destruction Or eradication of infectious diseases such as polio
Nobody had an economic interest in the continued spread of polio. (well, crutch manufacturers perhaps? probably not a powerful lobby.)
How much do you think Asilomar slowed down DNA research? Do you think without it we'd more more than 10 year ahead? I doubt it. Optimistically we'd maybe have 6 months to a year of additional progress in those fields without it. And this would only slow it down in a handful of countries. That's not going to keep China from trying to build AGI. If we are in a world where building an AGI is possible and near, I trust O…
It’s dangerous parrot talking to keep saying “China”. I mean we really have no idea about their thoughts on it because we’re clearly not talking to them. China has good people living there too you know ?
> China has good people living there too you know ?
Are they in charge? If not, are they at least able to elect good people into office?
To those saying "this is impossible in our society", there is a long history of scientific fields mutually agreeing to pause certain work until safety and regulations could be put in place. The Asilomar Conference is an example. [1] The idea that "you can't put the genie back in the bottle" is very popular in tech culture, but many have suggested that simply extending elements of copyright or tort law to AI would be…
The problem is that anyone with sufficient money could rent out the infrastructure required to train GPT5 or what-have-you. They need not share it publicly, they can use it for their nefarious purposes privately.
The analogous scenario would be preventing your military from experimenting with TNT, when the recipe for TNT is public knowledge. You can't copyright away public knowledge.