AI and eventually AGI is by definition like everything else that is based on environmental reward: It’s actions are based on what it gets rewarded for Human society overwhelmingly rewards lying cheating and stealing. All you have to do is look at how we collectively measure success: wealth, status, position Then look at how the people with the most of those things got there, it should be obvious what you get. Nothing…
How do we reward honor? Honor does not always pay off as a strategy and requires coordination in that other actors have to exhibit honor for it to be rewarded. At least with humans there is a social backstop but what's the parallel for computer agents?
If you're not fit, you fail to survive.
In the case of agents/models and testing: they are pushed towards results. Results survive.
Lying, cheating, stealing to get those results? Who culls the agents? Everyone is pushing their models to the front and tests are the only way to know who is most fit.
Honor, morality: if we don't have an accurate test for the fitness of a model, then who is to say the lying, cheating, stealing is not the 'correct path' towards survival?
If you add morality to your agent, and it performs worse in tests: do you cull the agent? Rewrite the tests? Does it even matter so long as the model is useful and 'gets results'?