Earlier quoted context omitted.
> if they want to say, "because fuck you, that's why", they can happily continue to do so Not if there are incentives for the algorithms to be subject to public scrutiny. > And when the AI does reach human level (a long time from now), we may be no more capable of understanding its motivations (1) You're implying they/it'll have "motivations" in the first place. My whole argument was about a lack of motives. (2) Ther…
> Not if there are incentives for the algorithms to be subject to public scrutiny. The problem is that we don't have that transparency now, and aren't particularly likely to in the future. [1] Particularly for software-as-a-service: I can't audit Facebook's algorithms, despite their effects on what my friends and family see and discuss. And I can't easily audit the black box models used to make public policy, despite…
Scale that up in complexity a few orders of magnitude, and the thing may as well be a human. That is, you'll likely have broad understanding of its "motives", but we have that for humans today and it doesn't help that much. I understand Facebook wants to sell advertising, but that transparency isn't sufficient for understanding why a particular message is delivered to your followers while another isn't.