"Read this story with a free account."
I'll pass thanks.
An AI wolf that preferred suicide over eating sheep
81–90 of 223 posts
Re: An AI wolf that preferred suicide over eating sheep
#82Re: An AI wolf that preferred suicide over eating sheep
#83Isn't this just a cock up with incentives? If they'd put a -100 score on dying it would have sorted itself out pretty quick.
Perhaps the PhD student wasn't trying to make an AI that wins at pac-man, but investigating something else. They mention "maximizing control over environment".
One of the most typical scenarios studied in those wolf/sheep models (like http://www.netlogoweb.org/launch#http://ccl.northwestern.edu... ) is to find the best conditions for "balance" between sheep and wolf: Too many wolves and the sheep go extint and later the wolf starve. Too many sheep and then the sheep don't get enough food and also die, taking the wolves with them..
Re: An AI wolf that preferred suicide over eating sheep
#84Similar story of unexpected AI outcomes... As part of my PhD research, I created a simplified Pac-Man style game where the agent would simply try to stay alive as long as possible whilst being chased by the 3 ghosts. The agent was un-motivated and understood nothing about the goal, but was optimising for maximising its observable control over the world (avoiding death is a natural outcome of this). I spent sometime t…
this sounds interesting. can you link your research or paper?
Re: An AI wolf that preferred suicide over eating sheep
#85Re: An AI wolf that preferred suicide over eating sheep
#86Interest in these click-bait type stories drops off dramatically for people who have ever implemented or even deeply thought about non-trivial models.
Re: An AI wolf that preferred suicide over eating sheep
#87Seems like a nothing story. Just looking at the game, there's obviously a constant decision to be made of chase more sheep or instantly die. It sounds like in the original model they had a max of 20 seconds, so it's not surprising that you would just tank your losses to maximize your score every now and then. Anyone who tries to devise optimal strategies for things should be able to see this isn't especially interest…
Exactly, from technical perspective it's a nothing story.
It's interesting, though, how strong of a reaction general public had to this. The story must have strongly resonated with what some folks were already feeling. When you squint (pretend to understand the technology not at all) it's a tragic story. The situation of the wolf seems similar to the situation of some people. Chasing their careers in a highly structured, sort of dehumanized, environment of constant pursuit. "Supreme Intelligence" (that's what a layperson may think of AI) looks at a situation of the wolf and decides that it makes no sense to continue the pursuit. Moreover, what is "optimal" is the most tragic result - suicide.
Re: An AI wolf that preferred suicide over eating sheep
#88One thing I've been considering: At what point does a creator have a moral or ethical obligation to a creation. Say you create an AI in a virtual world that keeps track of some sense of discomfort. How complex does the AI have to get to require some obligation? Just enough complexity to exhibit distress in a way to stir the creator's sympathy or empathy? The glib answer is never, of course. And one easy-out, I can th…
Utility functions are only defined up to addition of a constant and scaling by a positive constant. So instead of rewarding them with +5 and punishing them with -5, you can use 1005 and 995 instead. Problem solved.
The numbers are indeed arbitrary. But ultimately you want to avoid low utility/reward action and continue high utility/reward actions. That behavior, trying to avoid or pursue actions, would be indicative of the state of distress regardless of an arbitrary number attached to it.
Re: An AI wolf that preferred suicide over eating sheep
#89Similar story of unexpected AI outcomes... As part of my PhD research, I created a simplified Pac-Man style game where the agent would simply try to stay alive as long as possible whilst being chased by the 3 ghosts. The agent was un-motivated and understood nothing about the goal, but was optimising for maximising its observable control over the world (avoiding death is a natural outcome of this). I spent sometime t…
A while ago, a very simple agent I made had to do tasks in the maze and evaluate strategies to reach them. I wanted it to have no assumptions about the world, so it started with minimum knowledge. Its first plan was to try to remove walls, to get to the things it needed.
It is a fun feeling when your own program surprises you.
Re: An AI wolf that preferred suicide over eating sheep
#90Earlier quoted context omitted.
Yeah, because we have a -1000 points on death built-in.
Looking at genetic algorithms makes a great comparison. In essence any algorithm in which the wolf commits suicide doesn't make it to the next generation. It's the equivalent of an enormous score penalty and 100% analog to how it works for actual life.
Genetic algorithms are based on the same reward/cost function setup. They could easily arrive at the same conclusion because suicide might be the dominant strategy.