An AI wolf that preferred suicide over eating sheep • Hacker News Board

Re: An AI wolf that preferred suicide over eating sheep

#91

Post by wombatmobile » Tue, Jul 06, 2021, 2:32 PM UTC

The philosopher Hubert Dreyfus argued that computers, who have no body, no childhood and no cultural practice, could not acquire intelligence at all.

https://www.nature.com/articles/s41599-020-0494-4

What he means is that computers, which can learn rules and use those rules to make predictions in certain domains, nevertheless cannot exercise general intelligence because they are not "in the world". This renders them unable to experience and parse culture, most of which is tacit in real time, and sustained by enduring mental models which we experience as "expectations" that we navigate with our emotions and senses.

Culture is the platform on which intelligence is manifest, because the usefulness of knowledge is not absolute - it is contextual and social.

Re: An AI wolf that preferred suicide over eating sheep

#92

Post by zild3d » Tue, Jul 06, 2021, 2:34 PM UTC

FridayoLeary wrote: post #53

jhbadger wrote: post #4

I'm reminded of the fable (in Nick Bostrom's Superintelligence ) of the chess computer that ended up murdering anyone who tried to turn it off because in order to optimize winning chess games as programmed it has to be on and functional.

Which perverted mind would build into a chess computer the ability to kill?

Doesn't need a gun, just network access.

Re: An AI wolf that preferred suicide over eating sheep

#93

Post by wombatmobile » Tue, Jul 06, 2021, 2:36 PM UTC

markwkw wrote: post #87

spywaregorilla wrote: post #76

Seems like a nothing story. Just looking at the game, there's obviously a constant decision to be made of chase more sheep or instantly die. It sounds like in the original model they had a max of 20 seconds, so it's not surprising that you would just tank your losses to maximize your score every now and then. Anyone who tries to devise optimal strategies for things should be able to see this isn't especially interest…

Exactly, from technical perspective it's a nothing story. It's interesting, though, how strong of a reaction general public had to this. The story must have strongly resonated with what some folks were already feeling. When you squint (pretend to understand the technology not at all) it's a tragic story. The situation of the wolf seems similar to the situation of some people. Chasing their careers in a highly structu…

> The story must have strongly resonated with what some folks were already feeling.

Yes, because we don't see things as they are, we see them as we are.

Re: An AI wolf that preferred suicide over eating sheep

#94

Post by petercooper » Tue, Jul 06, 2021, 2:37 PM UTC

What are some of the nicest environments for experimenting with this sort of "define some rules, see how agents exist within that world" stuff? It doesn't need to be full on ML models, even simpler rules defined in code would be fine.

Re: An AI wolf that preferred suicide over eating sheep

#95

Post by johbjo » Tue, Jul 06, 2021, 2:37 PM UTC

TomAnthony wrote: post #10

Similar story of unexpected AI outcomes... As part of my PhD research, I created a simplified Pac-Man style game where the agent would simply try to stay alive as long as possible whilst being chased by the 3 ghosts. The agent was un-motivated and understood nothing about the goal, but was optimising for maximising its observable control over the world (avoiding death is a natural outcome of this). I spent sometime t…

It can depend on what the agent "sees" and how many time-steps away the "consequences" are. If the ghosts are so far away that any action will take t time-steps before consequences to the agent, the actions are pseudo-random because there is no reward to optimize on.

The number of outcomes in branching_factor^t (very large) makes the action-values at t=0 (where the agent chooses between two/three actions) almost uniform random.

Re: An AI wolf that preferred suicide over eating sheep

#96

Post by spywaregorilla » Tue, Jul 06, 2021, 2:41 PM UTC

markwkw wrote: post #87

spywaregorilla wrote: post #76

Seems like a nothing story. Just looking at the game, there's obviously a constant decision to be made of chase more sheep or instantly die. It sounds like in the original model they had a max of 20 seconds, so it's not surprising that you would just tank your losses to maximize your score every now and then. Anyone who tries to devise optimal strategies for things should be able to see this isn't especially interest…

Exactly, from technical perspective it's a nothing story. It's interesting, though, how strong of a reaction general public had to this. The story must have strongly resonated with what some folks were already feeling. When you squint (pretend to understand the technology not at all) it's a tragic story. The situation of the wolf seems similar to the situation of some people. Chasing their careers in a highly structu…

Shrug. Another way to frame this is a poker bot learned to fold when given a bad hand, and they only gave it the same bad hand.

Yes, yes, woe is the individual in modern capitalist society but the only reason people are reacting to this are that they don't understand it and they've been told it's something much more emotionally impactful than it actually is.

Re: An AI wolf that preferred suicide over eating sheep

#97

Post by queuebert » Tue, Jul 06, 2021, 2:43 PM UTC

This is the danger of not understanding what you're doing at a deep level.

Clearly in the (flawed) objective there is a phase transition near the very beginning, where the wolves have to chose whether to minimize the time penalty or maximize the score. With enough "temperature" and time perhaps they could transition to the other minimum, but the time penalty minimum is much closer to the initial conditions, so you know ab initio that it will be a problem. You can reduce that by making the time penalty much smaller than the sheep score and adding it only much later. I feel bad that the students wasted so much time on a badly formulated problem.

Edit: Also none of these problems are black boxes if you understand optimization. Knowing what is going on inside a very deep neural network (such as an AGI might have) is quite different than understanding the incentives created by a particular objective function.

Re: An AI wolf that preferred suicide over eating sheep

#98

Post by nabajour » Tue, Jul 06, 2021, 2:45 PM UTC

FridayoLeary wrote: post #53

jhbadger wrote: post #4

I'm reminded of the fable (in Nick Bostrom's Superintelligence ) of the chess computer that ended up murdering anyone who tried to turn it off because in order to optimize winning chess games as programmed it has to be on and functional.

Which perverted mind would build into a chess computer the ability to kill?

I think this comes from the theory of general artificial intelligence where your AI would have the ability for self improving. Hence it could develop any capability given time and incentive for it.

There are interesting videos on the subject on Robert Miles channel on AI safety: https://www.youtube.com/channel/UCLB7AzTwc6VFZrBsO2ucBMg

Re: An AI wolf that preferred suicide over eating sheep

#99

Post by arduinomancer » Tue, Jul 06, 2021, 2:46 PM UTC

This makes me wonder: is it possible for ML models to be provably correct?

Or is that completely thrown out the window if you use a ML model rather than a procedural algorithm?

Because if the model is a black box and you use it for some safety system in the real world, how do you know there isn’t some wierd combination of inputs that causes the model to exhibit bizzare behaviour?

Re: An AI wolf that preferred suicide over eating sheep

#100

Post by acituan » Tue, Jul 06, 2021, 2:46 PM UTC

markwkw wrote: post #87

spywaregorilla wrote: post #76

Seems like a nothing story. Just looking at the game, there's obviously a constant decision to be made of chase more sheep or instantly die. It sounds like in the original model they had a max of 20 seconds, so it's not surprising that you would just tank your losses to maximize your score every now and then. Anyone who tries to devise optimal strategies for things should be able to see this isn't especially interest…

Exactly, from technical perspective it's a nothing story. It's interesting, though, how strong of a reaction general public had to this. The story must have strongly resonated with what some folks were already feeling. When you squint (pretend to understand the technology not at all) it's a tragic story. The situation of the wolf seems similar to the situation of some people. Chasing their careers in a highly structu…

From the article in contrast to what you said;

> Perhaps the true lesson to be learnt here isn’t about helplessness and giving up. It’s about getting up, trying again and again, and staying with the story till the end.

I find the possibility of contrasting interpretations absurd. The problem with using any dead matter for our meaning making needs is it is ultimately a self-referential justification for how we think we should feel, while being equally or even more prone to self deception traps.

AI being the object is irrelevant here, this is nothing different than astrology or divination from tea leaves etc. It is 2000 BC level religious thinking with new toys.