Live data from Hacker News

An AI wolf that preferred suicide over eating sheep

lancengym.medium.com

181–190 of 223 posts

Re: An AI wolf that preferred suicide over eating sheep

#181

Earlier quoted context omitted.

What in the parent post is dualist? Sounds more like an argument that animals have embodied intelligence. But as for being a dualist in the 21st century, there is always consciousness, information and math. All three of which can lead to some form of dualism/platonism.

Many of Dreyfuss' and other similar arguments reduce do dualism when you start digging into them. I don't have the time to dig into the specific article, but here's some immediate questions: 1. What is special about a body that makes it impossible to have intelligence without it? (a) Is it possible for a quadriplegic person to be intelligent? (b) A blind and deaf person? ((c)What about that guy from Johnny Got His Gu…

> Many of Dreyfuss' and other similar arguments reduce do dualism when you start digging into them. I don't have the time to dig into the specific article, but here's some immediate questions:

To me it sounds dualist if intelligence is disembodied. If the substrate doesn't matter, only the functionality, then that sounds like there's something additional to the world than just the physical constintuents. But of course, embodied versions of intelligence need to answer the sort of questions you posed. It should be noticed that Dreyfuss wrote his objections in the 50s and 60s during the period of classical AI. I don't know whether he addressed the question of robot children, or simulated childhoods. We don't have the sort of thing even today, and we also don't have AGI. Some of his objections still stand, although machine learning and robotics research has made inroads.

> Math? Me, I'm a formalist. It's all a game that we've made up the rules to.

So why is physics so heavily reliant on mathematics? Quite a few physicists think the world has a mathematical structure.

> For example, "...there is always consciousness, information and math...": without a tight, and very technical, definition of consciousness, that seems to be assuming the conclusion.

Qualia would be the philosophical term for subjective experiences of color, sound, pain, etc. Reducing those to their material correlations has been notoriously difficult, and there is still no agreement on what that entails.

As for information, some scientists have been exploring the idea that chemical space leads to the emergence of information as an additional thing to physics which needs to be incorporated into our scientific understanding of the world. That we can't really explain biology without it.

Re: An AI wolf that preferred suicide over eating sheep

#182
post #160

Gwern has a list of similar stories: https://www.gwern.net/Tanks#alternative-examples

FWIW, I see a critical difference between OP and my reward hacking examples: OP is an example of how reward-shaping can lead to premature convergence to a local optima, which is indeed one of the biggest risks of doing reward-shaping - it'll slow down reaching the global optima rather than speeding it up, compared to the 'true' reward function of just getting a reward for eating a sheep and leaving speed implicit - b…

I think in some of your examples the global optimum might also have been the correct behaviour, it's just that the program failed to find it. For example the robot learning to use a hammer. It's hard to believe that throwing the hammer was just as good as using it properly.

Re: An AI wolf that preferred suicide over eating sheep

#184
post #62

One thing I've been considering: At what point does a creator have a moral or ethical obligation to a creation. Say you create an AI in a virtual world that keeps track of some sense of discomfort. How complex does the AI have to get to require some obligation? Just enough complexity to exhibit distress in a way to stir the creator's sympathy or empathy? The glib answer is never, of course. And one easy-out, I can th…

> The glib answer is never, of course

Dismissing “never” offhand without explanation is glib.

Re: An AI wolf that preferred suicide over eating sheep

#185

Earlier quoted context omitted.

Imagine being a dualist in the 21st century.

What in the parent post is dualist? Sounds more like an argument that animals have embodied intelligence. But as for being a dualist in the 21st century, there is always consciousness, information and math. All three of which can lead to some form of dualism/platonism.

> cannot exercise general intelligence because they are not "in the world".

Implies dualism. In a materialist world a computer can learn anything given the proper structure and stimuli.

Re: An AI wolf that preferred suicide over eating sheep

#187

Earlier quoted context omitted.

The issue with AI safety and unanticipated AI outcomes in general is that it’s always just a cock-up with incentives. It’s easy to sort out in narrowly specified areas, but an extremely hard problem as the tasks become more general.

Isn’t this true about all systems, not just “AI”? The definition of a software bug is an unintended behavior. In a large system, myriad intents overlap and combine in unexpected ways. You might imagine a complex enough system where the confidence that a modification doesn’t introduce an unintended behavior is near zero.

I think it’s true for many systems, not just AI that’s true.

AI is worth calling out in this regard because, if the field is successful enough, it can create dangerous systems that don’t behave how we want.

Building a safe general AI is much harder than building a general AI, which is why it’s worth considering AI as it’s own problem domain.

Re: An AI wolf that preferred suicide over eating sheep

#188
post #87

Seems like a nothing story. Just looking at the game, there's obviously a constant decision to be made of chase more sheep or instantly die. It sounds like in the original model they had a max of 20 seconds, so it's not surprising that you would just tank your losses to maximize your score every now and then. Anyone who tries to devise optimal strategies for things should be able to see this isn't especially interest…

Exactly, from technical perspective it's a nothing story. It's interesting, though, how strong of a reaction general public had to this. The story must have strongly resonated with what some folks were already feeling. When you squint (pretend to understand the technology not at all) it's a tragic story. The situation of the wolf seems similar to the situation of some people. Chasing their careers in a highly structu…

> pretend to understand the technology not at all

Are you missing a word or two?

Re: An AI wolf that preferred suicide over eating sheep

#190
Ok. Lots of AI stories here so I'll the best I've read, the student who trained an AI to work on upwork ;-)

https://news.ycombinator.com/item?id=5397797

Be sure to read to the end.

One of the answers is also pure gold in context:

> Don't feel bad, you just fell into one of the common traps for first-timers in strong AI/ML.

Post reply on HN