An AI wolf that preferred suicide over eating sheep • Hacker News Board

Re: An AI wolf that preferred suicide over eating sheep

#101

Post by ajuc » Tue, Jul 06, 2021, 2:46 PM UTC

markwkw wrote: post #87

spywaregorilla wrote: post #76

Seems like a nothing story. Just looking at the game, there's obviously a constant decision to be made of chase more sheep or instantly die. It sounds like in the original model they had a max of 20 seconds, so it's not surprising that you would just tank your losses to maximize your score every now and then. Anyone who tries to devise optimal strategies for things should be able to see this isn't especially interest…

Exactly, from technical perspective it's a nothing story. It's interesting, though, how strong of a reaction general public had to this. The story must have strongly resonated with what some folks were already feeling. When you squint (pretend to understand the technology not at all) it's a tragic story. The situation of the wolf seems similar to the situation of some people. Chasing their careers in a highly structu…

Similarly I've seen A LOT of people posting stories about "chat bot exposed to internet started praising Hitler and became racist/sexist/antisemitic" as a proof that "supreme intellect sees through leftist political correctness and knows that alt-right is correct about everything".

Re: An AI wolf that preferred suicide over eating sheep

#102

Post by rob74 » Tue, Jul 06, 2021, 2:49 PM UTC

spywaregorilla wrote: post #76

Seems like a nothing story. Just looking at the game, there's obviously a constant decision to be made of chase more sheep or instantly die. It sounds like in the original model they had a max of 20 seconds, so it's not surprising that you would just tank your losses to maximize your score every now and then. Anyone who tries to devise optimal strategies for things should be able to see this isn't especially interest…

It's not surprising from the perspective of an "AI actor". But if you call it a "wolf", most people will assume that it will behave at least roughly like a real-world creature, and the self-preservation instinct is one of the most basic traits of all living beings, so the "AI wolf" not having that is indeed surprising for a layperson.

Re: An AI wolf that preferred suicide over eating sheep

#103

Post by spywaregorilla » Tue, Jul 06, 2021, 2:53 PM UTC

RandomWorker wrote: post #37

https://www.bilibili.com/video/BV16X4y1V7Yu?p=1&share_medium... Here is the full video also linked at the bottom. It also shows the one that trained longer that the wolves start successfully hunting the sheep after more training examples.

The ai seems to die at the top of the map unexpectedly for some reason. Like 6:07.

Another interesting observation is that the wolves don't coordinate it seems. That probably implies that the reward functions are individual, so they're technically competing rather than cooperating.

Lastly... they seem to not be very good at the game even at the end

Re: An AI wolf that preferred suicide over eating sheep

#104

Post by joek1301 » Tue, Jul 06, 2021, 2:54 PM UTC

TomAnthony wrote: post #10

Similar story of unexpected AI outcomes... As part of my PhD research, I created a simplified Pac-Man style game where the agent would simply try to stay alive as long as possible whilst being chased by the 3 ghosts. The agent was un-motivated and understood nothing about the goal, but was optimising for maximising its observable control over the world (avoiding death is a natural outcome of this). I spent sometime t…

Yet another similar story. A side project of mine was building a rudimentary neural network whose weights were optimized via a genetic algorithm. The goal was operating top-down, 2D self-driving cars.

The cars' "fitness" function rewarded cars for driving along the course and punished them for crashing into walls. But evidently this function punished a little too severely: the most successful cars would just drive in tight circles and never make progress on the course. But they were sure to avoid walls. :)

Re: An AI wolf that preferred suicide over eating sheep

#105

Post by JoshTko » Tue, Jul 06, 2021, 2:55 PM UTC

Folks are missing why this went viral in China. From the article "In an even more philosophical twist, young and demoralized Chinese corporate citizens also saw the suicidal wolf as the perfect metaphor for themselves: a new class of white collar workers — often compelled to work ‘996' (9am to 9pm, six days a week) — chasing a dream of promotions, pay raise, marrying well… that seem to be becoming more and more elusive despite their grind."

Re: An AI wolf that preferred suicide over eating sheep

#106

Post by vnorilo » Tue, Jul 06, 2021, 2:59 PM UTC

spywaregorilla wrote: post #76

Seems like a nothing story. Just looking at the game, there's obviously a constant decision to be made of chase more sheep or instantly die. It sounds like in the original model they had a max of 20 seconds, so it's not surprising that you would just tank your losses to maximize your score every now and then. Anyone who tries to devise optimal strategies for things should be able to see this isn't especially interest…

If we play the analogy further: life is suffering, apart from the brief ecstasy of eating sheep. The AI was trying not to suffer, thus chose the boulder.

Did my best to translate the (misguided) fitness function to fiction.

Re: An AI wolf that preferred suicide over eating sheep

#107

Post by TomAnthony » Tue, Jul 06, 2021, 3:04 PM UTC

TchoBeer wrote: post #20

TomAnthony wrote: post #10

Similar story of unexpected AI outcomes... As part of my PhD research, I created a simplified Pac-Man style game where the agent would simply try to stay alive as long as possible whilst being chased by the 3 ghosts. The agent was un-motivated and understood nothing about the goal, but was optimising for maximising its observable control over the world (avoiding death is a natural outcome of this). I spent sometime t…

How did you measure control over the world?

The method was called 'empowerment'. Two ways to explain it...

From a mathematical perspective, we used Information Theory to model the world as an information theoretic 'loop'. The agent could 'send' a signal to the world by performing an action, which would change the state of the world; the state of the world was what the agent 'received'. This obviously relies on having a model of the world and what your actions will do, but doesn't burden the model with other biases.

Pore more colloquially, the agent could perform actions in the world, and see the resulting state of the world (in my case, that was the location of the agent and of the ghosts). Part of the principle was that changes you cannot observe are not useful to you.

Re: An AI wolf that preferred suicide over eating sheep

#108

Post by TomAnthony » Tue, Jul 06, 2021, 3:05 PM UTC

yodelshady wrote: post #31

TomAnthony wrote: post #10

Similar story of unexpected AI outcomes... As part of my PhD research, I created a simplified Pac-Man style game where the agent would simply try to stay alive as long as possible whilst being chased by the 3 ghosts. The agent was un-motivated and understood nothing about the goal, but was optimising for maximising its observable control over the world (avoiding death is a natural outcome of this). I spent sometime t…

I believe that tactic is called "kiting" and used by speedrunners?

Yes! Exactly - kiting. I didn't know the term but when I explained the behaviour I was seeing to a colleague they told me about this.

Re: An AI wolf that preferred suicide over eating sheep

#109

Post by TomAnthony » Tue, Jul 06, 2021, 3:07 PM UTC

johbjo wrote: post #95

TomAnthony wrote: post #10

Similar story of unexpected AI outcomes... As part of my PhD research, I created a simplified Pac-Man style game where the agent would simply try to stay alive as long as possible whilst being chased by the 3 ghosts. The agent was un-motivated and understood nothing about the goal, but was optimising for maximising its observable control over the world (avoiding death is a natural outcome of this). I spent sometime t…

It can depend on what the agent "sees" and how many time-steps away the "consequences" are. If the ghosts are so far away that any action will take t time-steps before consequences to the agent, the actions are pseudo-random because there is no reward to optimize on. The number of outcomes in branching_factor^t (very large) makes the action-values at t=0 (where the agent chooses between two/three actions) almost unif…

Yes, you are right.

I experimented with different time horizons, mostly look 3-7 steps ahead.

In terms of the 'reward', that was implicit within the model - if the ghosts caught you, your ability to influence the state of the world dropped to 0.

Re: An AI wolf that preferred suicide over eating sheep

#110

Post by TomAnthony » Tue, Jul 06, 2021, 3:09 PM UTC

fnord77 wrote: post #84

TomAnthony wrote: post #10

Similar story of unexpected AI outcomes... As part of my PhD research, I created a simplified Pac-Man style game where the agent would simply try to stay alive as long as possible whilst being chased by the 3 ghosts. The agent was un-motivated and understood nothing about the goal, but was optimising for maximising its observable control over the world (avoiding death is a natural outcome of this). I spent sometime t…

this sounds interesting. can you link your research or paper?

Sure! The PDF is available here:

https://uhra.herts.ac.uk/handle/2299/15376