Earlier quoted context omitted.
> I think most people will agree that a GAI can't be bargained with, tempted, bought or otherwise contained - We will be at its complete mercy regardless of any constraints we might think up. Who thinks this? I don't see any evidence that this is a common belief among people who work in the hard sciences related to AI, nor do I think it sound remotely logical It feels like some people are taking archetypes like pando…
A point Taleb keeps on making is that risk analysis is separate from the domain and shouldn't be done by domain experts. Global pandemic response plans for example shouldn't be done by virologists, because they are experts in viruses, not in how a pandemic which is a health/political/economic complex system behaves. The same way, AI risk plans shouldn't be done by AI researchers, just like we don't use neurologists f…
I think you need to have a proper bridge between the technical understanding and the people who manage the implications. In the case of longer standing diseases, for example, we're probably there. The risks are understood, and laypeople can weigh them as part of policy decisions. For new things like covid, we saw the world go crazy with misunderstanding, and ridiculous things like plexiglass barriers everywhere, and other talisman type stuff, as politicians tried to simultaneously abdicate responsibility to disease researchers, while grabbing at the parts they liked for political gains. But at least there was some grounding in reality because people do have a share and longstanding comprehension of disease spread and of the concepts of getting sick, etc.
New technology is the worst, because it gets blown up into some imagined concept that has no bearing on the reality. So, as I implied in the upstream comment, if we were on the verge of releasing some kind of sentient evil into the world, maybe the kind of silly speculation ("it can't be bargained with", etc) that basically rehashes Terminator, would be appropriate. But it's no more realistic than, say, the kid in Looper who has telekinetic powers and grows up to be an evil mob boss. It's just a made up bad thing that could happen, that if you talked to someone who knew the tech you'd realize is nonsense. That's very different from health threats we know exist.