I always thought that Asimov's Laws of Robotics ("A robot may not injure a human being" etc) were an interesting prop for science fiction, but wildly disconnected from the way computing & robotics actually work. Turns out he was just writing LLM prompts way ahead of his time.
Not only wildly disconnected, but purposefully created to show ambiguity of rules when interpreted by beings without empathy. All of Asimov's books that include the laws also include them being unintentionally broken through some edge-case.
I don’t think that’s the main problem, there are a lot of moral dilemmas where even humans can’t agree what’s right.