Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An AI wolf that preferred suicide over eating sheep

lancengym.medium.com

Post reply on HN
Print view View original

71–80 of 223 posts

Re: An AI wolf that preferred suicide over eating sheep

#71

Post by OscarCunningham »

rrmm wrote: post #62

One thing I've been considering: At what point does a creator have a moral or ethical obligation to a creation. Say you create an AI in a virtual world that keeps track of some sense of discomfort. How complex does the AI have to get to require some obligation? Just enough complexity to exhibit distress in a way to stir the creator's sympathy or empathy? The glib answer is never, of course. And one easy-out, I can th…

Utility functions are only defined up to addition of a constant and scaling by a positive constant. So instead of rewarding them with +5 and punishing them with -5, you can use 1005 and 995 instead. Problem solved.

Re: An AI wolf that preferred suicide over eating sheep

#72

Post by imtringued »

imtringued wrote: post #49
jonplackett wrote: post #17

Isn't this just a cock up with incentives? If they'd put a -100 score on dying it would have sorted itself out pretty quick.

No, it's a cock up with the source of the wolves. If you could respawn endlessly after death would you fear it? You'd just want the stupid game to end before you lose points from the timer.

For clarification purposes:

Let's say you are a human player playing the wolf and sheep game. The score achieved in the game decides your death in real life. Note the stark difference. Dying in the game is not the same thing as dying in real life.

If there is an optimal strategy in the game that involves dying in the game you are going to follow it regardless of whether you are a human or an AI. By adding an artificial penalty to death you haven't changed the behavior of the AI, you have changed the optimal strategy.

The human player and the AI player will both do the optimal strategy to keep themselves alive. For the AI "staying alive" doesn't mean staying alive in the game, it means staying alive in the simulation. Thus even a death fearing AI would follow the suicide strategy if that is the optimal strategy.

It is impossible conclude from the experiment whether the AI doesn't fear death and thus willingly commits suicide or whether it fears death so much that it follows an optimal strategy that involves suicide.

Re: An AI wolf that preferred suicide over eating sheep

#73

Post by inglor_cz »

lancengym wrote: post #28
McMiniBurger wrote: post #16

Earlier quoted context omitted.

hm... "keep your friends close but your enemies closer" ...?

But try to make sure your enemies don't end up surrounding you?

Yeah, that is tricky. I believe that Constantinople once found out the hard way, and thus is now Istanbul.

Re: An AI wolf that preferred suicide over eating sheep

#75

Post by xtracto »

abrahamneben wrote: post #63

This problem isn't particularly unique to AI research. In any optimization problem, if you do not encode all constraints or if your cost function does not always reflect the real world cost, then you will get incorrect or even nonsensical results. Describing this as an AI problem is just clickbait.

The article doesn't mention it but the researchers are using agent-based-modelling. It was nice to see the gif of what appears to be either NetLogo or Repast. I did research in that area for about 8 years and know a bit about the subject.

What they are showing is one of the main issues with agent-based-models (and I think every model, but it happens particularly with models trying to capture the behaviour of complex open systems): Garbage in -> Garbage Out.

Most likely the representation of the sheep/wolf system was not correct (so the modeling was not correct). Here "correct" means good enough to demonstrate whatever emerging behaviour they are studying. ABM is a powerful tool, but you must know how to use it.

Re: An AI wolf that preferred suicide over eating sheep

#76

Post by spywaregorilla »

Seems like a nothing story. Just looking at the game, there's obviously a constant decision to be made of chase more sheep or instantly die. It sounds like in the original model they had a max of 20 seconds, so it's not surprising that you would just tank your losses to maximize your score every now and then.

Anyone who tries to devise optimal strategies for things should be able to see this isn't especially interesting.

Social metaphors are wildly out of place.

They say "unintended consequences of a blackbox" but I doubt that's true. Make it a deterministic turn based game and run it through a perfectly transparent optimization model and I wouldn't be surprised to learn this was just the best strategy for the rules they devised. I really hate when people describe an ai as something that cannot be understood because they personally don't understand it.

Re: An AI wolf that preferred suicide over eating sheep

#77

Post by xtracto »

giantg2 wrote: post #74

For some reason this makes me think of corporate policies - how some people game them and how others except that the incentives are unattainable.

That's an interesting idea for an agent-based-model and a study: Show how certain corporate policies would push towards short term local-optima (what's happening in the article) instead of more long term global optimum states.

Re: An AI wolf that preferred suicide over eating sheep

#78

Post by Hackbraten »

Reminds me of this 2014 king-of-the-hill challenge: https://codegolf.stackexchange.com/questions/25347/survival-...

One particular solution stood out: https://codegolf.stackexchange.com/a/25357

The suicidal wolf became a (short-lived) running gag so it started appearing in other king-of-the-hill challenges: https://codegolf.stackexchange.com/a/34856

Re: An AI wolf that preferred suicide over eating sheep

#79

Post by fny »

spywaregorilla wrote: post #76

Seems like a nothing story. Just looking at the game, there's obviously a constant decision to be made of chase more sheep or instantly die. It sounds like in the original model they had a max of 20 seconds, so it's not surprising that you would just tank your losses to maximize your score every now and then. Anyone who tries to devise optimal strategies for things should be able to see this isn't especially interest…

If I remember correctly there were similar scenarios that would occur using that popular Berkeley Pacman universe where he would run into a ghost to avoid the penalty of living for too long.

Re: An AI wolf that preferred suicide over eating sheep

#80

Post by Retr0id »

TomAnthony wrote: post #10

Similar story of unexpected AI outcomes... As part of my PhD research, I created a simplified Pac-Man style game where the agent would simply try to stay alive as long as possible whilst being chased by the 3 ghosts. The agent was un-motivated and understood nothing about the goal, but was optimising for maximising its observable control over the world (avoiding death is a natural outcome of this). I spent sometime t…

Another similar story, I remember reading about an AI that simply paused the game when it was about to die. I can actually remember doing something similar as a child.
Post reply on HN