This just seems really obvious. Even if there are sheep nearby worth hunting, it's probably always eventually going to be the right move to suicide.
An AI wolf that preferred suicide over eating sheep
11–20 of 223 posts
Re: An AI wolf that preferred suicide over eating sheep
#12I remember a similar story about (I think) a Tetris game where the AI's training goal was to delay the Game Over screen as long as possible. So in the end the AI just paused the game indefinitely.
Re: An AI wolf that preferred suicide over eating sheep
#13I'm reminded of the fable (in Nick Bostrom's Superintelligence ) of the chess computer that ended up murdering anyone who tried to turn it off because in order to optimize winning chess games as programmed it has to be on and functional.
Interestingly I was just today explaining the paperclip optimizer scenario to a friend who asked about the dangers of AI, including the fact that there's almost no general optimization task that doesn't (with a sufficiently long lookahead) involve taking over the world as an intermediate step.
(Obviously closed, specific tasks like "land this particular rocket safely within 15 minutes" don't always lead to this, but open ended ones like "manufacture mcguffins" or "bring about world peace" sure seem to.)
Re: An AI wolf that preferred suicide over eating sheep
#14Reminds me of the old essay by 'Eliezer: "The Hidden Complexity of Wishes".
https://www.lesswrong.com/posts/4ARaTpNX62uaL86j6/the-hidden...
In it, there is a thought experiment of having an "Outcome Pump", a device that makes your wishes come true without violating laws of physics (not counting the unspecified internals of the device), by essentially running an optimization algorithm on possible futures.
As the essay concludes, it's the type of genie for which no wish is safe.
The way this relates to AI is by highlighting that even ideas most obvious to all of us, like "get my mother out of that burning building!", or "I want these virtual wolves to get better at eating these virtual sheep", carry incredible amount of complexity curried up in them - they're all expressed in context of our shared value system, patterns of thinking, models of the world. When we try to teach machines to do things for us, all that curried up context gets lost in translation.
Re: An AI wolf that preferred suicide over eating sheep
#15Get 80% OFF on CBS ALL ACCESS With a huge discount on a monthly subscription, CBS All Access unlimited by getting CBS ALL ACCESS coupons Code. CBS All Access is full of different plays to try out and become lost in, such as The Dusk Zones and Mysterious Angel, and Star Trek; Picard and the new take on The Show by Stephen King in 2020. Every account includes the ability to stream on two screens, and you can even access displays to stream in Offline Mode if you change from a No subscription. Users can best believe on CBS All Access Coupons if you already have a prime membership.
https://uttercoupons.com/front/store-profile/cbs-all-access-...
Re: An AI wolf that preferred suicide over eating sheep
#16Similar story of unexpected AI outcomes... As part of my PhD research, I created a simplified Pac-Man style game where the agent would simply try to stay alive as long as possible whilst being chased by the 3 ghosts. The agent was un-motivated and understood nothing about the goal, but was optimising for maximising its observable control over the world (avoiding death is a natural outcome of this). I spent sometime t…
hm... "keep your friends close but your enemies closer" ...?
Re: An AI wolf that preferred suicide over eating sheep
#17Isn't this just a cock up with incentives? If they'd put a -100 score on dying it would have sorted itself out pretty quick.
Re: An AI wolf that preferred suicide over eating sheep
#18[deleted]
Re: An AI wolf that preferred suicide over eating sheep
#19Reminds me of the old essay by 'Eliezer: "The Hidden Complexity of Wishes". https://www.lesswrong.com/posts/4ARaTpNX62uaL86j6/the-hidden... In it, there is a thought experiment of having an "Outcome Pump", a device that makes your wishes come true without violating laws of physics (not counting the unspecified internals of the device), by essentially running an optimization algorithm on possible futures. As the essay…
Interesting essay. I think the big blind spot for humans programming AI is also the fact that we tend to overlook the obvious, whereas algorithms will tend to take the path of least resistance without prejudice or coloring by habit and experience.
Re: An AI wolf that preferred suicide over eating sheep
#20Similar story of unexpected AI outcomes... As part of my PhD research, I created a simplified Pac-Man style game where the agent would simply try to stay alive as long as possible whilst being chased by the 3 ghosts. The agent was un-motivated and understood nothing about the goal, but was optimising for maximising its observable control over the world (avoiding death is a natural outcome of this). I spent sometime t…
How did you measure control over the world?