Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An AI wolf that preferred suicide over eating sheep

lancengym.medium.com

Post reply on HN
Print view View original

211–220 of 223 posts

Re: An AI wolf that preferred suicide over eating sheep

#211

Post by taneq »

NaturalPhallacy wrote: post #209
taneq wrote: post #65

Earlier quoted context omitted.

We're just optimizing (generally quite well, I might add) for genetic survival.

Ian Banks did a really amazing exposition of this where the Culture was rallying to stamp out reproducing nanites and they had to be stopped because if not they'd literally turn the whole universe into copies of themselves. One of the human characters mused that isn't that what all life is trying to do? I think it was in the Hydrogen Sonata, but I'm not sure.

Yes! I often find myself thinking of organisms as 'hegemonising swarms.'

Re: An AI wolf that preferred suicide over eating sheep

#212

Post by wombatmobile »

TchoBeer wrote: post #206
wombatmobile wrote: post #199

Earlier quoted context omitted.

Don't overthink this. If you put a computer in a room with a hot babe, a 3 layer chocolate cake, a bottle of the finest whisky or bourbon, the keys to a Porsche, and a trillion dollars in cash, what would it do? Yeah, nothing. The computer is not in the world.

What if we build a computer that would do something with those things? Additionally, if I care about neither food nor drink nor money nor cars, am I not in the world?

> I care about neither food nor drink nor money nor cars, am I not in the world?

Assuming you are human, that depends on how long you care not about food or drink.

Re: An AI wolf that preferred suicide over eating sheep

#213

Post by TomAnthony »

benlivengood wrote: post #171
TomAnthony wrote: post #112

Earlier quoted context omitted.

The approach I used was similar. The idea of maximising observed control of the world means you seek states where you can reach many other states, but _predictably_ so. This comes 'for free' when using Information Theory to model a channel.

What's the tradeoff between "delete all state in the world with 100% certainty" and "be able to choose any next state of the world with (100-epsilon)% certainty"?

In Information Theory, there is a concept of Channel Capacity. If a channel is defined as the probability of the output being s if you send a, across all possible values of a, then the Channel Capacity is the maximum amount of information you can communicate across this channel, measured in bits.

To achieve the Channel Capacity you need to find the optimum distribution across a - i.e. what set of signals maximises the information you can transmit on this channel. There are known algorithms for finding this distribution (e.g. Blahut-Arimoto).

Now if you model the world as a channel, where s represents the reachable states and a represents the actions the agent can take (and the channel, P(s|a), represents the dynamics of the world), you can calculate what actions allow you maximal control (in terms of states you can controllably reach).

More info in this paper: https://uhra.herts.ac.uk/handle/2299/15376

Re: An AI wolf that preferred suicide over eating sheep

#214

Post by goatlover »

colinmhayes wrote: post #185
goatlover wrote: post #123

Earlier quoted context omitted.

What in the parent post is dualist? Sounds more like an argument that animals have embodied intelligence. But as for being a dualist in the 21st century, there is always consciousness, information and math. All three of which can lead to some form of dualism/platonism.

> cannot exercise general intelligence because they are not "in the world". Implies dualism. In a materialist world a computer can learn anything given the proper structure and stimuli.

Yes, but the key is the proper structure and stimuli, which animal intelligence gets through having a body. Can we get computers to have the same sort of intelligence without a synthetic body? This becomes more of a robotics versus traditional AI debate. Think Rodney Brooks versus Marvin Minsky.

Re: An AI wolf that preferred suicide over eating sheep

#215

Post by goatlover »

colinmhayes wrote: post #203
wombatmobile wrote: post #201

Earlier quoted context omitted.

> (a) Is it possible for a quadriplegic person to be intelligent? (b) A blind and deaf person? Yes of course, because all of those people have ambitions and desires. They feel pain and they seek pleasure, which they experience through their bodies. Imagine if the world 2,000 years from now was populated only by supercomputers, all the lifeforms having perished. What are these computers going to do with the planet?

Why can't a computer have ambitions and desires? Why can't it seek pleasure and feel pain? The only answer is dualism or we don't know how to wire it properly yet.

Or we don't have the proper design. If we want machines to be like animals, maybe we need to make them that way. Like the replicants in Blade Runner, or the humanoid "toasters" in the recent Battlestar Galactica.

Re: An AI wolf that preferred suicide over eating sheep

#216

Post by lancengym »

SubiculumCode wrote: post #130
markwkw wrote: post #87

Earlier quoted context omitted.

Exactly, from technical perspective it's a nothing story. It's interesting, though, how strong of a reaction general public had to this. The story must have strongly resonated with what some folks were already feeling. When you squint (pretend to understand the technology not at all) it's a tragic story. The situation of the wolf seems similar to the situation of some people. Chasing their careers in a highly structu…

Exactly. It is a social commentary story where a result from a student's project was a lucid analogy of the plight of their lived rat-race in modern China, with the lesson being: Cut your losses and lie flat. To those within ML field, this is less than new, but as a commentary on how such ML issues can be a teachable and easily understood analogy to people's lives certainly makes the story interesting to me.

Spot on. Funny results from poorly specified AI experiments have cropped up since the 90's. But the interestingly angel her is how this one came out of nowhere at the right time and resonated with young working class Chinese.

Re: An AI wolf that preferred suicide over eating sheep

#217

Post by lancengym »

spywaregorilla wrote: post #193
colinmhayes wrote: post #119

Earlier quoted context omitted.

>but the only reason people are reacting to this are that they don't understand it I think it's much more likely that they're reacting like this because they see their own plight in the wolf. It doesn't matter why the wolf killed itself, it became a meme that allowed many Chinese to reflect together on a common plight.

Having reread the article, it seems like the concept of suicide doesn't weigh into the cultural reaction at all. It's just giving up on the chase.

Yes you are quite right. The social media reactions did not suggest an attitude of suicide at all. It was more of laid back life instead of meeting expectations and attaining so-called success.

Re: An AI wolf that preferred suicide over eating sheep

#218

Post by lancengym »

kzrdude wrote: post #66
m12k wrote: post #29

I think a major takeaway here is that balancing a reward system to reward more than a single behavior is really hard - it's easy to tip the scales so one behavior completely dominates all others. It's an interesting lens to use to look at the heuristic reward system humans have built in (hunger, fear, desire, etc). This tends to have an adaptation/numbing effect, where repeated rewards of the same type tend to have d…

leela (lc0) chess also has this problem. People sometimes thinks it wins too slowly (prefers some surefire way to win by 50 moves instead of slightly more risky by 5 moves), or that it plays without tact when in a losing position (it's hard for it to rank moves when all of them lead to a loss, it doesn't have the sense that humans do of still preserving the beauty of the game). AIs need to learn to feel awkward and a…

What do you make of the documentary on AlphaGo where the AI did seemingly suicidal and incomprehensible moves to the human masters but won in the end, baffling everyone? https://youtu.be/WXuK6gekU1Y

Re: An AI wolf that preferred suicide over eating sheep

#220

Post by ajmurmann »

spywaregorilla wrote: post #90
ajmurmann wrote: post #59

Earlier quoted context omitted.

Looking at genetic algorithms makes a great comparison. In essence any algorithm in which the wolf commits suicide doesn't make it to the next generation. It's the equivalent of an enormous score penalty and 100% analog to how it works for actual life.

Genetic algorithms are based on the same reward/cost function setup. They could easily arrive at the same conclusion because suicide might be the dominant strategy.

Yes, definitely. I misread the parent comment as snark claiming that we don't have that score penalty in reality
Post reply on HN