How to Build a Computer Model of God

Suppose one was trying to train up a batch of true AIs that were generous, moral, loving, responsible, principled, etc. (think Santa, Jesus, or Fred Rogers). I would submit that the task would be much better suited to some combination of genetic programming and machine learning i.e. generate a bunch of AIs with varying characteristics and let them learn for a while and see how they do at the stated task. I think it would make a lot of sense for a simulation scientist to want to have a bunch of these around, they could do some really cool stuff for you and you wouldn't have to worry too much about IRB approval since they would be certified moral by the previous test.

What kind of test would make sense? It would have to be one with difficult moral choices, one with an opportunity for failure. One without unnecessary pain and stress, but one that allows the AIs to make moral mistakes and be held responsible for them. If the true AIs are interacting, and it makes sense that they probably would have to, then naturally the duds could make life a little miserable for the good ones, you'd want to minimize that as much as you could without restricting the ability of all AIs to choose freely among available opportunities given to them. On top of that you might put them in a hostile environment that requires work and effort for survival to see if they have what it takes to get stuff done.

Now if that doesn't sound like life I don't know what does. I that context hell doesn't really make a lot of sense (would you really punish bad AIs forever or just put them in charge of non-morally challenging and easy jobs? (my first thought was the coffee pot, but maybe putting a misbehaving AI in charge of that is not such a good idea) and "heaven" becomes more of a "gets to do cool, interesting stuff" than "chill out playing harp". From my experience many religious people tend to punt on what actually an afterlife would be for, other than it would be nice to reward us for being good. The answer to "Why can't the simulation master give the nice heaven to everybody?" is pretty clear in this context. She can't because that would be really bad for whatever task she needed moral AI for, which, if it affected a bunch of other true AIs has huge moral implications. So sorry, no offence, but based on your test results you don't get to do the cool stuff.

As Dijkstra said, "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim."

Somehow, I don't think most religious people (with exceptions like Deists) will feel comfortable with that possibility ;)

Which should be a big honking, flashing neon sign that you're both arguing without any facts to back you up, ie you're both just making stuff up. You might as well argue about whether Santa has white or black fur as the trim on his coat, for all of the utility that you will get out of such arguments.