Is Ethical A.I. Even Possible?

However, the issue is that it's very culturally dependent on WHY it's seen as wrong, and it's the reason behind WHY something is seen as wrong that the more complex elements of ethics are built on.

A utilitarian vs a deontologist vs a virtue theorist vs a follower of almost any religion vs a supporter of god knows how many other moral theories I don't know about would all have a different answer to that, and different answers to which cases killing or stealing or anything else might potentially be justified/right.

And you can see that reflected in legal systems of different countries right now. Some countries have self defence as basically 'anything is permissible if they're intruding on your property' where some have it so you need to use reasonable force. Some countries consider it fair to let the government kill criminals via the death penalty, and others don't.

The difficulty isn't defining whether murder or theft or what not in its most blatant forms is wrong, since most ethical frameworks will state it is. It's trying to define the many, many edge cases that people don't agree on, and which many countries/states/societies take different approaches to.

   No it's not. There are cases where taking a human life is considered acceptable and ethical. Killing an abductor that threatens to kill many people and negotiations have failed  is considered ethical and not punishable by law.
   Now if you assume that murder is the case where an unlawful act happened then you've stopped querying for ethics and your question becomes what is lawful which is again extremely ill-defined.

Does it?

At a personal level, I'm much more scared of a bad person with a gun and access to me than a bad person with an AI and access to me. It's more important than the ethical use of a teacup, but certainly rivaled and beaten at a micro level.

At a macro level I think your argument has more ground to stand on when it comes to things like improved efficiency of mass surveillance, but how much of that is AI part rather than the mass surveillance part? What immoralities are enabled/accelerated that could not be done without AI? In the end I'm just as concerned about mass surveillance as I was before. In a practical sense, even if somehow passed laws limiting AI, do I really think that a government performing mass surveillance in the shadows is going to follow those laws when AI is a concept anyone (with the know-how) can implement? I don't think AI's power/capacity is as high as people think, it just tends to fit well with some very bad macro level ethical actions to enhance them.

I'm not sure I'm convinced either way yet, but the claim of "unrivaled" made me immediately skeptical, at least considering the AI of this decade(s). In a hundred years you're probably right with the unrivaled part, though atom bombs and the like are probably close seconds.

Or a fax machine.

Or broccoli.

Or a pencil.

(Yeah, you could stab someone in the eye with the latter, but it's not something inherently violent about a pencil more than 2000 other things you could do it with, and most people would never do anything like that despite having used one).

Other things however, are either precisely made for harm and profit (e.g. a grenade, or "Hot Pockets"), or lend themselves very well to it (e.g. dynamite, internet cookies, etc.).

That's a religious view of AI and humans, where humans have special qualities (like soul and consciousness) perhaps evolved "magically" or "god given", that a machine can't have.

There are no serious arguments why an AI can't have perfect human-like consciousness, feelings and everything.

Inversely, there are no serious arguments why humans are special in any way, and their brain mappings can't be replicated by technical AI (eg. artificial neurons) or emulated by software.