This isn't really a story about AI; it's a story about an incompetent engineer, and the way that humans are quick to assign consciousness and emotions to things. Children believe their dolls are real, people comfort their roombas when there's a storm, and systems keep passing the Turing test because people want to be fooled. Ever since Eliza [1], and probably before, we've ascribed intelligence to machines because we…
The book, that is… not the movie.
The moral error in the book was humans wrongly thinking machines were sentient.
The movie turned this theme in its head, where the moral error was humans failing to appreciate the machines were sentient.
Anyway, I always found the book’s theme was far more interesting.
Great harm can come from humans attributing sentience to an unthinking inanimate object. It’s human nature to fall into this odd quagmire, be it idolatry, omens, astrology, and even silicon sex dolls…
Creepy.