Earlier quoted context omitted.
AI can also completely liberate all of us. This fear mongering is irrational. We already live in an extremely advanced reality, with organic life having organic nanomachines performing incredibly complex operations in a universe, in realtime, performing impossibly complex calculations. Even to simulate a square meter of our universe in realtime is outlandishly beyond our compute access. Perhaps AI unleashes the creat…
> Just because someone has the background and knowledge to build something, does not mean they have the philosophical chops nor the aptitude to understand it, or understand how to properly use it. Yes, but this is an argument against your position, not for it. It is deeply worrying that some software people are so immersed in the “move fast and break things” that they think it’s reasonable to race on no matter what’s…
This is the actual problem, that we've been trained to think that there are no consequences to trying everything ever and we will always be able to pick ourselves up after failure, until one day we find we can't anymore.
Sadly this line of thinking does apply to a lot of personal problems. People often say that you can't always succeed, that you can only move forward with failures under your belt to have first-hand experience of what not to do. That we have to experience failure first in order to stop ourselves from making bad decisions, otherwise we remain clueless.
But the stakes are so much higher at this scale, and the damage can become irreparable with a single mistake. It's to the point where we take as a given that people will just keep pressing on, without even exploring the reasons why we do, and why it's inevitable.
Pure ridiculous speculation: If needing to see the experience of destroying ourselves is necessary to stop us from destroying ourselves, a kind of weird theory has been floating around in my head related to that. Civilizations on the brink of collapse after they've moved too fast and broke themselves for the last time scrounge together their collective knowledge and shoot it off into space so that future civilizations can interpret them. Preferably in a way that makes encountering the time capsule seem like the word of some deity. The following civilizations would treat it as the word of God to prevent their own collapse, without the hard evidence of experimenting with to the point where they cannot resist developing it into an unstoppable superweapon.
It would be like the golden record on the Voyager, except instead of communicating "we exist, and here's what our planet is like", it would go more like "we exist, and here's how to not end up like us."