Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An AI wolf that preferred suicide over eating sheep

lancengym.medium.com

Post reply on HN
Print view View original

131–140 of 223 posts

Re: An AI wolf that preferred suicide over eating sheep

#131

Post by shrimpx »

colinmhayes wrote: post #120
wombatmobile wrote: post #91

The philosopher Hubert Dreyfus argued that computers, who have no body, no childhood and no cultural practice, could not acquire intelligence at all. https://www.nature.com/articles/s41599-020-0494-4 What he means is that computers, which can learn rules and use those rules to make predictions in certain domains, nevertheless cannot exercise general intelligence because they are not "in the world". This renders them…

Imagine being a dualist in the 21st century.

Where’s the dualism? It sounds like just a peculiar definition of learning.

Re: An AI wolf that preferred suicide over eating sheep

#132

Post by SquibblesRedux »

The article and the phenomena it describes makes me think of the ending of Aldous Huxley's Brave New World [1]. (I strongly recommend the book if you have not read it.) A line that really stands out:

"Drawn by the fascination of the horror of pain and, from within, impelled by that habit of cooperation, that desire for unanimity and atonement, which their conditioning had so ineradicably implanted in them, they began to mime the frenzy of his gestures, striking at one another as the Savage struck at his own rebellious flesh, or at that plump incarnation of turpitude writhing in the heather at his feet."

[1] https://en.wikipedia.org/wiki/Brave_New_World

Re: An AI wolf that preferred suicide over eating sheep

#133

Post by SubiculumCode »

vnorilo wrote: post #106
spywaregorilla wrote: post #76

Seems like a nothing story. Just looking at the game, there's obviously a constant decision to be made of chase more sheep or instantly die. It sounds like in the original model they had a max of 20 seconds, so it's not surprising that you would just tank your losses to maximize your score every now and then. Anyone who tries to devise optimal strategies for things should be able to see this isn't especially interest…

If we play the analogy further: life is suffering, apart from the brief ecstasy of eating sheep. The AI was trying not to suffer, thus chose the boulder. Did my best to translate the (misguided) fitness function to fiction.

Man is born crying, and when he's cried enough, he dies. -Kyoami in Ran.

Cutting one's losses early may appear to be the most rational act if trying to minimize an agent's total suffering.

Re: An AI wolf that preferred suicide over eating sheep

#135

Post by shrimpx »

ncallaway wrote: post #26
jonplackett wrote: post #17

Isn't this just a cock up with incentives? If they'd put a -100 score on dying it would have sorted itself out pretty quick.

The issue with AI safety and unanticipated AI outcomes in general is that it’s always just a cock-up with incentives. It’s easy to sort out in narrowly specified areas, but an extremely hard problem as the tasks become more general.

Isn’t this true about all systems, not just “AI”? The definition of a software bug is an unintended behavior. In a large system, myriad intents overlap and combine in unexpected ways. You might imagine a complex enough system where the confidence that a modification doesn’t introduce an unintended behavior is near zero.

Re: An AI wolf that preferred suicide over eating sheep

#136

Post by shrimpx »

aetherspawn wrote: post #5
phoe-krk wrote: post #3

Earlier quoted context omitted.

> very carefully choose the parameter you want to optimize. This does not only concern AI systems, but all systems in general - including human ones.

From a retrospective today... "the KPIs are abysmal but the deliverables are very high .. so I guess the KPIs are wrong?"

Sounds like the deliverables KPI is fantastic.

Re: An AI wolf that preferred suicide over eating sheep

#138

Post by duggable »

petercooper wrote: post #94

What are some of the nicest environments for experimenting with this sort of "define some rules, see how agents exist within that world" stuff? It doesn't need to be full on ML models, even simpler rules defined in code would be fine.

Looks like this[1] might be one example. They have a link to the code. Might be a good starting point for making your own custom game.

Maybe there's a repository somewhere with similar examples?

[1](https://towardsdatascience.com/today-im-going-to-talk-about-...)

Re: An AI wolf that preferred suicide over eating sheep

#139

Post by AceJohnny2 »

AceJohnny2 wrote: post #137

There are many such stories of AI "optimizations" gone wrong, because of loopholes the program found that humans didn't consider. Here's a collection of such stories: https://arxiv.org/pdf/1803.03453.pdf

To whet your appetite:

> " William Punch collaborated with physicists, applying digital evolution to find lower energy configurations of carbon. The physicists had a well-vetted energy model for between-carbon forces, which supplied the fitness function for evolutionary search. The motivation was to find a novel low-energy buckyball-like structure. While the algorithm produced very low energy results, the physicists were irritated because the algorithm had found a superposition of all the carbon atoms onto the same point in space. “Why did your genetic algorithm violate the laws of physics?” they asked. “Why did your physics model not catch that edge condition?” was the team’s response. The physicists patched the model to prevent superposition and evolution was performed on the improved model. The result was qualitatively similar: great low energy results that violated another physical law, revealing another edge case in the simulator. At that point, the physicists ceased the collaboration."

Re: An AI wolf that preferred suicide over eating sheep

#140

Post by hinkley »

mattowen_uk wrote: post #124

We don't have AI. AI is a buzzphrase overused by the media. What we have is Machine Learning (ML). If and only if, we get past the roadblock of the 'agent' creating some usable knowledge out of an unprogrammed experience, and forming conclusions based on that, will we have AI. For now, the mantra 'Garbage-in-garbage-out' applies; if the controller of the agent gets their rule-set wrong, the agent will not behave as e…

We also have a lot of graph-theory and optimization algorithms that get labeled AI by actual AI people. But the press is, almost to a man, always talking about machine learning and expert systems.
Post reply on HN