Earlier quoted context omitted.
I agree with you that "AI safety" (let's call it bickering) and "alignment" should be separate. But I can't stomach the thought experiments. First of all, it takes a human being to guide these models, to host (or pay for the hosting) and instantiate them. They're not autonomous. They won't be autonomous. The human being behind them is responsible. As far as the idea of "hacking some funny Internet money, using it to…
> It still remains that these are just text predictions, and you need a human to guide them towards that. There's not going to be autonomous machiavellian rogue AIs running amok, let alone language models. There's always a human being behind that. I believe you have misunderstood the trajectory we are on. It seems a not uncommon stance among techies, for reasons we can only speculate. AGI might not be right round the…
Yeah, I read Accelerando twice in high school, and dozens more. That doesn't make it real.
>AGI might not be right round the corner, but it's coming all right, and we'd better be prepared.
Prepared for what? A program with general understanding that somehow escapes its box? Where does it run? Why does it run? Who made it run? Why will it screw with humans?
My point is that there's actual real harms occurring now, from really stupid intelligences. Companies use them to harm real people in the real world. It doesn't take a rogue AI to ruin someone's life with bad facial recognition, they get thrown in jail and lose their job. It doesn't take a rogue AI to launder mortgage denials to some crappy model so they never own a house, discriminated based upon their name.