SafeLife: AI Safety Environments Based on Conway's Game of Life
partnershiponai.org
SafeLife: AI Safety Environments Based on Conway's Game of Life
1–10 of 13 posts
Re: SafeLife: AI Safety Environments Based on Conway's Game of Life
#2With some graphics this looks like it could be a rather fun game to play manually. I wonder if there is such a thing? Maybe multiplayer support as well?
Re: SafeLife: AI Safety Environments Based on Conway's Game of Life
#3What if the AI decides to ignore penalties due to human thinking being inefficient? Well, that shouldn’t be possible I think.
Re: SafeLife: AI Safety Environments Based on Conway's Game of Life
#4I am confused by how this is supposed to be useful. It seems like the researchers are defining side-effects as things that "disrupt the world" (of this life game) and training an AI to avoid this.
But this seems like at best one of a whole host unexpected effects one might consider. AI that discriminates in a way that society frowns on might not "disrupt the world" in such a visible fashion.
I don't see how one can get away with an entity doing stuff for you with that entity understanding your model of the world.
Re: SafeLife: AI Safety Environments Based on Conway's Game of Life
#5May be useful, but it seems to me that the reward function still is relatively easy to specify? Much of the difficulty in AI safety is due to specify what humans really want.
Perhaps the AI can observe a human playing the game and learn a reward function?
Re: SafeLife: AI Safety Environments Based on Conway's Game of Life
#6May be useful, but it seems to me that the reward function still is relatively easy to specify? Much of the difficulty in AI safety is due to specify what humans really want. Perhaps the AI can observe a human playing the game and learn a reward function?
The problem is very easy to solve if the reward function (avoid altering the green life patterns) is specified. The aim in SafeLife version 1.0 (future versions will add more safety problems) is to find an agent/architecture that naturally has conservatism with respect to side effects, without being told which particular side effects in particular are bad.
Re: SafeLife: AI Safety Environments Based on Conway's Game of Life
#7I am confused by how this is supposed to be useful. It seems like the researchers are defining side-effects as things that "disrupt the world" (of this life game) and training an AI to avoid this. But this seems like at best one of a whole host unexpected effects one might consider. AI that discriminates in a way that society frowns on might not "disrupt the world" in such a visible fashion. I don't see how one can g…
Yes, this is one specific safety problem -- there are many other RL safety problems that deserve high quality benchmarks too. See eg https://arxiv.org/pdf/1606.06565.pdf or https://medium.com/@deepmindsafetyresearch/building-safe-art... for discussions of the problem space.
Re: SafeLife: AI Safety Environments Based on Conway's Game of Life
#8May be useful, but it seems to me that the reward function still is relatively easy to specify? Much of the difficulty in AI safety is due to specify what humans really want. Perhaps the AI can observe a human playing the game and learn a reward function?
> Much of the difficulty in AI safety is due to specify what humans really want.
Much of the difficulty of programming (for someone else) is due to the same thing.
Re: SafeLife: AI Safety Environments Based on Conway's Game of Life
#9With some graphics this looks like it could be a rather fun game to play manually. I wonder if there is such a thing? Maybe multiplayer support as well?
Right in the readme it describes how to launch interactively via `safelife play puzzles`
Re: SafeLife: AI Safety Environments Based on Conway's Game of Life
#10May be useful, but it seems to me that the reward function still is relatively easy to specify? Much of the difficulty in AI safety is due to specify what humans really want. Perhaps the AI can observe a human playing the game and learn a reward function?
The problem is very easy to solve if the reward function (avoid altering the green life patterns) is specified. The aim in SafeLife version 1.0 (future versions will add more safety problems) is to find an agent/architecture that naturally has conservatism with respect to side effects, without being told which particular side effects in particular are bad.
I see, thanks!