Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An AI wolf that preferred suicide over eating sheep

lancengym.medium.com

Post reply on HN
Print view View original

171–180 of 223 posts

Re: An AI wolf that preferred suicide over eating sheep

#171

Post by benlivengood »

TomAnthony wrote: post #112
greenpresident wrote: post #48

Earlier quoted context omitted.

In an active inference approach you would have the agent minimise surprisal. Choose the action that is most likely to produce the outcome you predicted.

The approach I used was similar. The idea of maximising observed control of the world means you seek states where you can reach many other states, but _predictably_ so. This comes 'for free' when using Information Theory to model a channel.

What's the tradeoff between "delete all state in the world with 100% certainty" and "be able to choose any next state of the world with (100-epsilon)% certainty"?

Re: An AI wolf that preferred suicide over eating sheep

#172

Post by SubiculumCode »

syntheticnature wrote: post #166
SubiculumCode wrote: post #133

Earlier quoted context omitted.

Man is born crying, and when he's cried enough, he dies. -Kyoami in Ran. Cutting one's losses early may appear to be the most rational act if trying to minimize an agent's total suffering.

David Benatar reached a similar philosophic conclusion due to his utilitarian views, which was amusingly put (with a sort of AI present, no less) in this webcomic: https://existentialcomics.com/comic/253

Thanks. I think I just found a new comic to read.

Re: An AI wolf that preferred suicide over eating sheep

#173

Post by cornel_io »

I mean, lesson zero of optimization is when you're designing a loss function and trying to incentivize agents to perform a task, don't set it up so that suicide has a higher payoff than making partial progress on the task. Maybe make death the worst outcome, not one of the best...?

One of these days I have to actually scour the web and collect a few good examples where evolutionary methods are used effectively on problems that actually benefit from them, assuming I can find them. Almost every example you're likely to see is either a) solved much more effectively by a more traditional approach like normal gradient descent or classic control theory techniques (most physical control experiments fall into this category), b) poorly implemented because of crappy reward setup, c) fully mutation-driven and hence missing what is actually good about evolution above and beyond gradient descent (crossover), or d) using such a trivial genotype to phenotype mapping that you could never hope to see any benefit from evolutionary methods beyond what gradient descent would give you (if the genome is a bunch of neural network weights, you're definitely in this category).

Re: An AI wolf that preferred suicide over eating sheep

#174

Post by spywaregorilla »

988747 wrote: post #146
SubiculumCode wrote: post #133

Earlier quoted context omitted.

Man is born crying, and when he's cried enough, he dies. -Kyoami in Ran. Cutting one's losses early may appear to be the most rational act if trying to minimize an agent's total suffering.

Which is why some forms of Buddhism are basically a cult of death: https://en.wikipedia.org/wiki/Sokushinbutsu

> In the video game The Legend of Zelda: Breath of the Wild, the monks in the Ancient Shrines seem to be based on sokushinbutsu.

Factoid of the day for sure

Re: An AI wolf that preferred suicide over eating sheep

#175

Post by 0110101001 »

Retr0id wrote: post #80
TomAnthony wrote: post #10

Similar story of unexpected AI outcomes... As part of my PhD research, I created a simplified Pac-Man style game where the agent would simply try to stay alive as long as possible whilst being chased by the 3 ghosts. The agent was un-motivated and understood nothing about the goal, but was optimising for maximising its observable control over the world (avoiding death is a natural outcome of this). I spent sometime t…

Another similar story, I remember reading about an AI that simply paused the game when it was about to die. I can actually remember doing something similar as a child.

https://youtu.be/xOCurBYI_gY&t=15m10s

Re: An AI wolf that preferred suicide over eating sheep

#176

Post by cpach »

cperciva wrote: post #150
wombatmobile wrote: post #91

The philosopher Hubert Dreyfus argued that computers, who have no body, no childhood and no cultural practice, could not acquire intelligence at all. https://www.nature.com/articles/s41599-020-0494-4 What he means is that computers, which can learn rules and use those rules to make predictions in certain domains, nevertheless cannot exercise general intelligence because they are not "in the world". This renders them…

The philosopher Hubert Dreyfus argued that computers, who have no body, no childhood and no cultural practice, could not acquire intelligence at all. Similarly, nuclear submarines, which lacking all of the critical organs of fish, are completely unable to swim.

The nuclear submarine is just part of our extended phenotype :)

Re: An AI wolf that preferred suicide over eating sheep

#177

Post by er4hn »

markwkw wrote: post #87
spywaregorilla wrote: post #76

Seems like a nothing story. Just looking at the game, there's obviously a constant decision to be made of chase more sheep or instantly die. It sounds like in the original model they had a max of 20 seconds, so it's not surprising that you would just tank your losses to maximize your score every now and then. Anyone who tries to devise optimal strategies for things should be able to see this isn't especially interest…

Exactly, from technical perspective it's a nothing story. It's interesting, though, how strong of a reaction general public had to this. The story must have strongly resonated with what some folks were already feeling. When you squint (pretend to understand the technology not at all) it's a tragic story. The situation of the wolf seems similar to the situation of some people. Chasing their careers in a highly structu…

> Exactly, from technical perspective it's a nothing story.

I think that one thing it points to is how technology can discover novel iterations on a system. Imagine if this was a system modeled around a network and the agent was trying to figure out how to get from the outside to read a specific system asset. With the right (read: very detailed) modeling you could create a pentesting agent.

Re: An AI wolf that preferred suicide over eating sheep

#178

Post by benlivengood »

FridayoLeary wrote: post #53
jhbadger wrote: post #4

I'm reminded of the fable (in Nick Bostrom's Superintelligence ) of the chess computer that ended up murdering anyone who tried to turn it off because in order to optimize winning chess games as programmed it has to be on and functional.

Which perverted mind would build into a chess computer the ability to kill?

A human mind not giving due consideration to the effects of granting arbitrarily high intelligence to an agent with simplistic morality counter to human morality.

From there it's a sequence of steps that would show up in a thorough root cause analysis ("humanity, the postmortem") where the agent capitalizes on existing abilities to gain more abilities until murder is available to it. It would likely start small with things like noticing the effects of stress or tiredness or confusion on human opponents and seeking to exploit those advantages by predicting or causing them, requiring more access to the real world not entirely represented by a chess board.

Re: An AI wolf that preferred suicide over eating sheep

#179

Post by sitkack »

BoxOfRain wrote: post #165
edoceo wrote: post #118

Earlier quoted context omitted.

At a Grateful Dead show in Oakland this geezer said to me: Your perception IS your reality man!

I'd have loved to have been arond to a Dead show! I know it sounds a little ungrateful coming from someone who lives in a period of unprecedented access to all kinds of wonderful music being written all the time, but there's something about the Dead that really connects with me that I can't quite put my finger on.

Dark Star Orchestra is your current best bet https://www.youtube.com/watch?v=y8_THRZLSi4

Re: An AI wolf that preferred suicide over eating sheep

#180

Post by benlivengood »

Brendinooo wrote: post #39
jonplackett wrote: post #17

Isn't this just a cock up with incentives? If they'd put a -100 score on dying it would have sorted itself out pretty quick.

I think the point is more about highlighting the fact that AI doesn't share our base assumptions. We wouldn't think to put a huge penalty on dying because humans generally think that death is bad.

Humans don't put a huge penalty on dying. We discount it and assume/pretend that once we've had a good long life then death is okay and euthanasia is preferable to suffering with no hope of recovery. AI wolves that can live for 20 seconds are unwilling to suffer -1 per second with no hope of sheep.
Post reply on HN