I don't know much about the philosophy of AI and I'm only familiar at a basic level with modern AI algorithms. From what I have been exposed to I don't see any reason to think AI is any more than a set of statistical frameworks. Is there any reason to believe that these statistical frameworks are comparable to biological intelligence? Am I thinking about this the wrong way?
It sounds like you're talking about the kinds of AI in use today? That's not what the cautions are about, since current "AI" however good at reading individual words or flying drones is not yet capable of human-level thought. The cautions are about trans-sapient AI, which doesn't exist yet. Even if it's simply a beefed up "set of statistical frameworks" linked in the right way to get a computer behave like a human, h…
I can see how a bug in those kinds of systems could cause things like that without necessarily being conscious or intelligent.