Wouldn't a higher penalty for bolder hits solve that problem, especially a high penalty for suicide?
Would be more realistic because dying has higher cost than failing.
An AI wolf that preferred suicide over eating sheep
41–50 of 223 posts
Re: An AI wolf that preferred suicide over eating sheep
#42Isn't this just a cock up with incentives? If they'd put a -100 score on dying it would have sorted itself out pretty quick.
I think the point is more about highlighting the fact that AI doesn't share our base assumptions. We wouldn't think to put a huge penalty on dying because humans generally think that death is bad.
Yeah, because we have a -1000 points on death built-in.
Re: An AI wolf that preferred suicide over eating sheep
#43Reminds me of the old essay by 'Eliezer: "The Hidden Complexity of Wishes". https://www.lesswrong.com/posts/4ARaTpNX62uaL86j6/the-hidden... In it, there is a thought experiment of having an "Outcome Pump", a device that makes your wishes come true without violating laws of physics (not counting the unspecified internals of the device), by essentially running an optimization algorithm on possible futures. As the essay…
Aesop managed to make the point a lot more concisely: "Be careful what you wish for, lest it come true." (Although now that I look, I don't think that's a translation of any specific part of the text.)
Yes, but that moral is attached to a story. Morals and saws work as handles - they're useful for communication if both you and your interlocutor know the thing they're pointing to. Conversely, they are of little use until you read the story from which the moral comes, or personally experience the thing the saw talks about.
Re: An AI wolf that preferred suicide over eating sheep
#44Earlier quoted context omitted.
Interesting essay. I think the big blind spot for humans programming AI is also the fact that we tend to overlook the obvious, whereas algorithms will tend to take the path of least resistance without prejudice or coloring by habit and experience.
Yes. What I like about AI research is that it teaches us about all the things we take for granted, it shows us just how much of meaning is implicit and built on shared history and circumstances.
The hard part about programming is that you have to tell the computer what you want it to do.
Re: An AI wolf that preferred suicide over eating sheep
#45Just remember that you are optimizing for what you actually encoded in your rewards, your system, and your evaluation procedure, not for what narrative you constructed about what you think you are doing.
I had my own expeirience with this when I tried to train "rat" to get out of the maze. I rewarded rats for exiting but for some simple labirynths I generated for testing it was possible to exit it by just going straight ahead. So this strategy quickly dominated my testing population.
Re: An AI wolf that preferred suicide over eating sheep
#46Earlier quoted context omitted.
Yes. What I like about AI research is that it teaches us about all the things we take for granted, it shows us just how much of meaning is implicit and built on shared history and circumstances.
The hard part about programming is that you have to tell the computer what you want it to do.
The difficult, but in many ways rewarding, core of that is that it forces you to finally figure out what you actually want, because the computer won't accept anything except perfect clarity.
Re: An AI wolf that preferred suicide over eating sheep
#47Isn't this just a cock up with incentives? If they'd put a -100 score on dying it would have sorted itself out pretty quick.
While obviously I've got the advantage of hindsight here, it seems like it should not have taken three days of analysis to see why the wolves were committing suicide. It seems obvious once the point system is explained. Perhaps some rubber-duck debugging might have helped in this case.
Re: An AI wolf that preferred suicide over eating sheep
#48Similar story of unexpected AI outcomes... As part of my PhD research, I created a simplified Pac-Man style game where the agent would simply try to stay alive as long as possible whilst being chased by the 3 ghosts. The agent was un-motivated and understood nothing about the goal, but was optimising for maximising its observable control over the world (avoiding death is a natural outcome of this). I spent sometime t…
How did you measure control over the world?
In an active inference approach you would have the agent minimise surprisal. Choose the action that is most likely to produce the outcome you predicted.
Re: An AI wolf that preferred suicide over eating sheep
#49Isn't this just a cock up with incentives? If they'd put a -100 score on dying it would have sorted itself out pretty quick.
No, it's a cock up with the source of the wolves. If you could respawn endlessly after death would you fear it? You'd just want the stupid game to end before you lose points from the timer.
Re: An AI wolf that preferred suicide over eating sheep
#50Dev: it’s a bug
Manager to boss: It’s a crazy new AI behaviour that is going viral around the world!