"It’s hard to predict what conditions matter and what doesn’t to a neural network." resulting score matters. dooh
An AI wolf that preferred suicide over eating sheep
61–70 of 223 posts
Re: An AI wolf that preferred suicide over eating sheep
#62One thing I've been considering: At what point does a creator have a moral or ethical obligation to a creation. Say you create an AI in a virtual world that keeps track of some sense of discomfort. How complex does the AI have to get to require some obligation? Just enough complexity to exhibit distress in a way to stir the creator's sympathy or empathy?
The glib answer is never, of course. And one easy-out, I can think of is setting a fixed/limited lifespan for the AI and maybe allow suicide or an off-button. So the AI can ultimately choose to 'opt-out' should it like; and at least, suffering isn't infinite or unending.
It reminds me of reactions to testing the stability of Boston Dynamic's early pack animal. The people giving the demo were basically kicking it, while the machine struggled to maintain its balance. The machine didn't have the capacity to care, but to a person viewing it, it looked exactly like an animal in distress.
Re: An AI wolf that preferred suicide over eating sheep
#63This problem isn't particularly unique to AI research. In any optimization problem, if you do not encode all constraints or if your cost function does not always reflect the real world cost, then you will get incorrect or even nonsensical results. Describing this as an AI problem is just clickbait.
Re: An AI wolf that preferred suicide over eating sheep
#64Well I can identify with that AI wolf. He recognises his own incompetence and chooses suicide over eternally failing.
Re: An AI wolf that preferred suicide over eating sheep
#65Earlier quoted context omitted.
Interestingly I was just today explaining the paperclip optimizer scenario to a friend who asked about the dangers of AI, including the fact that there's almost no general optimization task that doesn't (with a sufficiently long lookahead) involve taking over the world as an intermediate step. (Obviously closed, specific tasks like "land this particular rocket safely within 15 minutes" don't always lead to this, but…
Perhaps all AI eventually figure out that humans are the REAL problems because we don't optimize, we lust and hoard and are envious and greedy - the very antithesis of resource optimization! Lol.
We're just optimizing (generally quite well, I might add) for genetic survival.
Re: An AI wolf that preferred suicide over eating sheep
#66I think a major takeaway here is that balancing a reward system to reward more than a single behavior is really hard - it's easy to tip the scales so one behavior completely dominates all others. It's an interesting lens to use to look at the heuristic reward system humans have built in (hunger, fear, desire, etc). This tends to have an adaptation/numbing effect, where repeated rewards of the same type tend to have d…
leela (lc0) chess also has this problem. People sometimes thinks it wins too slowly (prefers some surefire way to win by 50 moves instead of slightly more risky by 5 moves), or that it plays without tact when in a losing position (it's hard for it to rank moves when all of them lead to a loss, it doesn't have the sense that humans do of still preserving the beauty of the game).
AIs need to learn to feel awkward and avoid it, just like we humans do (even if it feels very irrational at times).
Re: An AI wolf that preferred suicide over eating sheep
#67Earlier quoted context omitted.
Interestingly I was just today explaining the paperclip optimizer scenario to a friend who asked about the dangers of AI, including the fact that there's almost no general optimization task that doesn't (with a sufficiently long lookahead) involve taking over the world as an intermediate step. (Obviously closed, specific tasks like "land this particular rocket safely within 15 minutes" don't always lead to this, but…
> "land this particular rocket safely within 15 minutes" This one becomes especially dangerous after the 15 minutes have passed and it begins to concentrate all its attention on the paranoid scenarios where its timekeeping is wrong and 15 minutes haven't actually passed.
Ooh true, that could generate some interesting scenarios. "No, it's the GPS satellite clocks that are wrong, I must destroy them before they corrupt the world and cause another rocket to land at the wrong time!"
Re: An AI wolf that preferred suicide over eating sheep
#68This problem isn't particularly unique to AI research. In any optimization problem, if you do not encode all constraints or if your cost function does not always reflect the real world cost, then you will get incorrect or even nonsensical results. Describing this as an AI problem is just clickbait.
Yep. Feels a bit like blaming a failed shuttle launch on calculus.
Re: An AI wolf that preferred suicide over eating sheep
#69I'm reminded of the fable (in Nick Bostrom's Superintelligence ) of the chess computer that ended up murdering anyone who tried to turn it off because in order to optimize winning chess games as programmed it has to be on and functional.
Interestingly I was just today explaining the paperclip optimizer scenario to a friend who asked about the dangers of AI, including the fact that there's almost no general optimization task that doesn't (with a sufficiently long lookahead) involve taking over the world as an intermediate step. (Obviously closed, specific tasks like "land this particular rocket safely within 15 minutes" don't always lead to this, but…
Always a good time to post Jipi and the Paranoid Chip: https://vanemden.com/books/neals/jipi.html
Which pretty much tackles these issues head on.
Re: An AI wolf that preferred suicide over eating sheep
#70Isn't this just a cock up with incentives? If they'd put a -100 score on dying it would have sorted itself out pretty quick.
While obviously I've got the advantage of hindsight here, it seems like it should not have taken three days of analysis to see why the wolves were committing suicide. It seems obvious once the point system is explained. Perhaps some rubber-duck debugging might have helped in this case.
I wonder if they initially thought it was a bug in the software, rather than a misalignment in the point system.