Earlier quoted context omitted.
> The problem is we have as yet struggled to clearly articulate consistently what our actual best interests are, in terms of goals we can train into our AIs. And imho, we never will be able to do that, because as soon as there is more than one human, they are likely to disagree about something. Even things that should be no-brainers like "should we preserve our habitat or burn it down for profit" , or questions like…
> Even things that should be no-brainers like "should we preserve our habitat or burn it down for profit", or questions like "is it a good idea to have loads of deadly assault weapons just float around in our society", seem to be too hard for our species to resolve. These are brain intensive questions because they require deep moral, political, economic and ecological context. It's very difficult to express such deep…
They really are not. Even rodents with brains the size of a peanut manage to not willfully destroy the environment they are living in, despite already having all the resources they need, out of sheer greed.