Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An AI wolf that preferred suicide over eating sheep

lancengym.medium.com

Post reply on HN
Print view View original

161–170 of 223 posts

Re: An AI wolf that preferred suicide over eating sheep

#161

Post by cmehdy »

TomAnthony wrote: post #112
greenpresident wrote: post #48

Earlier quoted context omitted.

In an active inference approach you would have the agent minimise surprisal. Choose the action that is most likely to produce the outcome you predicted.

The approach I used was similar. The idea of maximising observed control of the world means you seek states where you can reach many other states, but _predictably_ so. This comes 'for free' when using Information Theory to model a channel.

Do you have any reading you'd recommend related to this?

I naively thought it would be some kind of Kalman filtering of sorts but from what I gather in your words it doesn't even have to be "that" complicated, right?

edit: found your link to the paper in another post ( https://news.ycombinator.com/item?id=27749619 ), thanks!

Re: An AI wolf that preferred suicide over eating sheep

#162

Post by mcguire »

goatlover wrote: post #123
colinmhayes wrote: post #120

Earlier quoted context omitted.

Imagine being a dualist in the 21st century.

What in the parent post is dualist? Sounds more like an argument that animals have embodied intelligence. But as for being a dualist in the 21st century, there is always consciousness, information and math. All three of which can lead to some form of dualism/platonism.

Many of Dreyfuss' and other similar arguments reduce do dualism when you start digging into them. I don't have the time to dig into the specific article, but here's some immediate questions:

1. What is special about a body that makes it impossible to have intelligence without it? (a) Is it possible for a quadriplegic person to be intelligent? (b) A blind and deaf person? ((c)What about that guy from Johnny Got His Gun?)

2. What is special about a childhood such that a machine cannot have it?

3. Would a person transplanted into a completely alien culture not be intelligent?

What is fundamentally being argued is the definition of "intelligence", and there are many fixed points of those arguments. Unfortunately, most of them (such as those that answer "no", "probably not", and "definitely not" to 1a, 1b, and 1c) don't really satisfy the intuitive meaning of "intelligence". That, and the general tone of the arguments, seem to imply the only acceptable meaning is dualism.

For example, "...there is always consciousness, information and math...": without a tight, and very technical, definition of consciousness, that seems to be assuming the conclusion. With a tight, and very technical, definition of consciousness, what is the problem with a machine demonstrating it?

Information? Check out knowledge, "justified true belief", and the Gettier problem (https://courses.physics.illinois.edu/phys419/sp2019/Gettier....).

Math? Me, I'm a formalist. It's all a game that we've made up the rules to.

Re: An AI wolf that preferred suicide over eating sheep

#163

Post by spywaregorilla »

phkahler wrote: post #148
spywaregorilla wrote: post #76

Seems like a nothing story. Just looking at the game, there's obviously a constant decision to be made of chase more sheep or instantly die. It sounds like in the original model they had a max of 20 seconds, so it's not surprising that you would just tank your losses to maximize your score every now and then. Anyone who tries to devise optimal strategies for things should be able to see this isn't especially interest…

It's good because most people can understand it. I'd say it's a perfect strategy for a game, but if they're using evolutionary algorithms they should require some form of reproduction for the wolves to carry on. That would make the suicide strategy fail to propagate well. I can also see a number of possible strange outcomes even then.

You're conflating the evolution of the strategy with the idea of the evolution of the actor being controlled by the agent. To give an obvious example, if dying gave 100 points instead of subtracting 10, even the dumbest evolutionary algo would learn to commit suicide asap. The survival of the actor has no intrinsic relevance to how the evolution develops.

Re: An AI wolf that preferred suicide over eating sheep

#165

Post by BoxOfRain »

edoceo wrote: post #118
wombatmobile wrote: post #93

Earlier quoted context omitted.

> The story must have strongly resonated with what some folks were already feeling. Yes, because we don't see things as they are, we see them as we are.

At a Grateful Dead show in Oakland this geezer said to me: Your perception IS your reality man!

I'd have loved to have been arond to a Dead show! I know it sounds a little ungrateful coming from someone who lives in a period of unprecedented access to all kinds of wonderful music being written all the time, but there's something about the Dead that really connects with me that I can't quite put my finger on.

Re: An AI wolf that preferred suicide over eating sheep

#166

Post by syntheticnature »

SubiculumCode wrote: post #133
vnorilo wrote: post #106

Earlier quoted context omitted.

If we play the analogy further: life is suffering, apart from the brief ecstasy of eating sheep. The AI was trying not to suffer, thus chose the boulder. Did my best to translate the (misguided) fitness function to fiction.

Man is born crying, and when he's cried enough, he dies. -Kyoami in Ran. Cutting one's losses early may appear to be the most rational act if trying to minimize an agent's total suffering.

David Benatar reached a similar philosophic conclusion due to his utilitarian views, which was amusingly put (with a sort of AI present, no less) in this webcomic: https://existentialcomics.com/comic/253

Re: An AI wolf that preferred suicide over eating sheep

#167

Post by spywaregorilla »

mcguire wrote: post #149
spywaregorilla wrote: post #76

Seems like a nothing story. Just looking at the game, there's obviously a constant decision to be made of chase more sheep or instantly die. It sounds like in the original model they had a max of 20 seconds, so it's not surprising that you would just tank your losses to maximize your score every now and then. Anyone who tries to devise optimal strategies for things should be able to see this isn't especially interest…

" I really hate when people describe an ai as something that cannot be understood because they personally don't understand it. " On the other hand, keep in mind that a significant weakness of most modern AI research is that it's extremely difficult to understand: you have the input, the output, and a bag of statistical weights. In the story, you know the (trivially bad) function that is being optimized; in general yo…

The tooling for understanding complex models is a lot better than what most people assume.

> The initial bizarre wolf behavior was simply the result of ‘absolute and unfeeling rationality’ exhibited by AI systems.

This is a bad quote. They should not say this. It's a poorly trained agent doing a decent job of a poorly defined environment. Absolute rationality conjures images of some greater thinking but its actually a really stupid model that hit a local maxima. Calling it unfeeling implies the model has some concept of "wolf" and "suicide" but it does not. Replace the visuals with single pixel dots if you want an honest depiction of the room for feelings.

> It’s hard to predict what conditions matter and what doesn’t to a neural network."

This is generally true, but it isn't true here.

Re: An AI wolf that preferred suicide over eating sheep

#168

Post by eldenbishop »

nojs wrote: post #116
TeMPOraL wrote: post #14

Reminds me of the old essay by 'Eliezer: "The Hidden Complexity of Wishes". https://www.lesswrong.com/posts/4ARaTpNX62uaL86j6/the-hidden... In it, there is a thought experiment of having an "Outcome Pump", a device that makes your wishes come true without violating laws of physics (not counting the unspecified internals of the device), by essentially running an optimization algorithm on possible futures. As the essay…

Related to the paperclip maximiser [1]: > Suppose we have an AI whose only goal is to make as many paper clips as possible. The AI will realize quickly that it would be much better if there were no humans because humans might decide to switch it off. Because if humans do so, there would be fewer paper clips. Also, human bodies contain a lot of atoms that could be made into paper clips. The future that the AI would be…

There is a wonderful little game based on this concept called universal paperclips. The AI eventually consumes all the matter in the universe in order to turn it into paperclips.

https://www.decisionproblem.com/paperclips/

Re: An AI wolf that preferred suicide over eating sheep

#169

Post by BoxOfRain »

ajuc wrote: post #101
markwkw wrote: post #87

Earlier quoted context omitted.

Exactly, from technical perspective it's a nothing story. It's interesting, though, how strong of a reaction general public had to this. The story must have strongly resonated with what some folks were already feeling. When you squint (pretend to understand the technology not at all) it's a tragic story. The situation of the wolf seems similar to the situation of some people. Chasing their careers in a highly structu…

Similarly I've seen A LOT of people posting stories about "chat bot exposed to internet started praising Hitler and became racist/sexist/antisemitic" as a proof that "supreme intellect sees through leftist political correctness and knows that alt-right is correct about everything".

It's really not that deep, people will always find sport in scandalising people with a stronger disgust reaction than themselves. It's more a new way of teaching a parrot to say "fuck" rather than a heartfelt statement of political belief in my opinion.

Re: An AI wolf that preferred suicide over eating sheep

#170

Post by spywaregorilla »

hotwire wrote: post #154
fny wrote: post #79

Earlier quoted context omitted.

If I remember correctly there were similar scenarios that would occur using that popular Berkeley Pacman universe where he would run into a ghost to avoid the penalty of living for too long.

It reminds me of the thread about the Quake 3 bots, who left alone for several years, figured out that the best approach was to not kill each other. https://i.imgur.com/dx7sVXj.jpg

Without knowledge of their reward function its difficult to tell if they're converged on this strategy or if its just broken.
Post reply on HN