Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thoughts on OpenAI, reinforcement learning, and killer robots

fast.ai

Post reply on HN
Print view View original

121–130 of 205 posts

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#121

Post by [deleted] »

jph00 wrote: post #36
_qbjt wrote: post #28

> It is hard for me to empathize with Musk’s fixation on evil super-intelligent AGI killer robots in a very distant future. I would argue this person has no business working with AI with this sort of myopic thinking. If you think "fake news" is a problem now, just wait until decentralized AI networks are able to tweet, publish articles and affect public discourse in a way that is indistinguishable from human influenc…

Oh please. The issue isn't what the end result could look like, but whether we'll actually get there if basic societal needs aren't met first. Prioritize things which are needed now and in the near future.

[deleted]

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#122

Post by throw2016 »

_qbjt wrote: post #28

> It is hard for me to empathize with Musk’s fixation on evil super-intelligent AGI killer robots in a very distant future. I would argue this person has no business working with AI with this sort of myopic thinking. If you think "fake news" is a problem now, just wait until decentralized AI networks are able to tweet, publish articles and affect public discourse in a way that is indistinguishable from human influenc…

AI is far away if at all. Many are trying to pass off image recognition and pattern matching used to identify pornographic images as AI and this just reflects their tenuous grasp of the problem at hand. Something like the human brain is well beyond this level of thinking.

Advancements if any will inevitably come from academia and those who have devoted decades studying the human brain. Not software engineers intoxicated by themselves.

Nothing in the current software ecosystem gives you the tools or language to conceptualize or even imagine let alone write code that can qualify to be used in the context of ai. Code only does what it is programmed to do. If else is a decision, that doesn't make it ai, multiple it by a million times and its still not ai.

This does not mean it cannot be dangerous. Autonomous weapons, anything thing to do with image recognition will find uses, but this is nothing like the what people imagine when we talk about AI. Engineers are supposed to be precise in their use of language but AI proponents are currently engaged neck deep in hype. Self driving cars will happen on extremely constrained roads, like trains. That will be the wake up call for the excitable ones to come back down to earth.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#123

Post by gone35 »

peferron wrote: post #61
j2kun wrote: post #43

Earlier quoted context omitted.

My issue is that, while of course you're right that one can't know for sure, some people conclude that simply not being able to know justifies whipping up a frenzy and (in all likelihood) wasting part of a billion dollars that could go to, say, more productive research even within AI/ML. Anyone could make a long list of things it's impossible to know for sure that would destroy humnaity. Many such things are vastly m…

Ok, I'll bite. Can you list several things that are vastly more likely to lead to the complete extinction of the human race (I'll assume that's what you meant by "destroy humanity") than malicious AGI?

- Asteroid strike

- Superbug pandemic

- Supervolcano eruption

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#124

Post by pixl97 »

eastWestMath wrote: post #114
nopinsight wrote: post #91

Earlier quoted context omitted.

The point is whether we should be complacent and dismiss concerns just because we don't know when it will happen. Once someone builds it, stopping it might be very difficult. Here's why: https://youtu.be/4l7Is6vOAOA (less than 9 minutes and very clearly explained). Dismissing even a 10% chance of possible catastrophic risks is not what we practice in any other domains. Would you dismiss a concern over airplanes that…

I guess, as a theoretical computer scientist myself, my first reaction is to roll my eyes. I get that these guys are trying to drum up interest with investors/funding agencies, but "My greatest fear is that my research is too successful" is still pretty grandiose. It sounds like those physicists who thought they had figured everything out in the '30s (or AI researchers before the last winter). I think it's pretty tel…

Zuckerberg just wants to use ANI to better push ads at you. Any fear of AI will make it harder for him to do just that. This is all about mone and earning more of it.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#125

Post by PeterisP »

NortySpock wrote: post #88
peferron wrote: post #61

Earlier quoted context omitted.

Ok, I'll bite. Can you list several things that are vastly more likely to lead to the complete extinction of the human race (I'll assume that's what you meant by "destroy humanity") than malicious AGI?

Climate Change, leading to frequent crop failures, leading to the collapse of most nations and most industrial capacity. That's what keeps me up at night these days.

The question was about complete extinction. A population reduction by 90% is a horrible disaster, but it sets us back "just" a couple centuries, but it's hardly something that can eliminate humanity permanently; unlike quite a few other things.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#126

Post by jph00 »

backpropaganda wrote: post #115
jph00 wrote: post #103

Earlier quoted context omitted.

There's nothing hard to understand about RL. I'm not sure where you get that idea - if you find it hard, perhaps you just need to look at some different way. Karpathy summarizes the differences with regular supervised learning in his policy gradient post: > Policy gradients is exactly the same as supervised learning with two minor differences: 1) We don’t have the correct labels yi so as a “fake label” we substitute…

I find RL hard and no I don't need Karpathy's or fast.ai's dumbed down version of RL. I'm talking about cutting-edge RL like PSRL or RETRACE, not policy gradient or DQN.

Well OK... I don't know why you think those are hard (let alone cutting edge - PSRL isn't new), but it seems important for you to feel like you understand things other people aren't smart enough to, so there's probably not much more I can say.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#127

Post by j2kun »

_qbjt wrote: post #113
j2kun wrote: post #99

Earlier quoted context omitted.

You are imagining super intelligent malicious AI (or AI piloted by malicious actors) with orders of magnitude more resources and reach than we have today is instantaneously implanted into today's society. That is also not relevant to the future implications of AI, because as AI evolves so will humanity. If you demand how exactly this will come about, then I will first require you to explain exactly how superintellige…

> That is also not relevant to the future implications of AI, because as AI evolves so will humanity. So why do black hat hackers exist? Why do you have so much faith in humanity's ability and/or willingness to "evolve" alongside AI? My prediction is that AI will evolve and humans will still be the same murderous creatures they are today, only with more tools at their disposal. > You're also ignoring basic ideas that…

Why do you have so much faith that autonomous superhuman AI will be such a Dalek-eqsue threat? If it's faith against faith, and you have no evidence to back up your side (while humanity adapting has millennia of evidence), there's no point in discussing it.

> Humans are pretty easy to manipulate...My fear is that with greater capabilities,...

Nobody is disagreeing with you. But this is hardly an observation of the apocalyptic proportions warned by AI risk proponents. If you're just suggesting governments regulate AI use, then you've changed your position from "stop the impending doom" to "limit the use of a dangerous tool." I have no doubt that when AI becomes good enough to be a dangerous tool, it will cause some major event and be regulated thereafter. You should start with that instead of "robots killing humans."

But if your argument is actually "stop the impending doom," then I disagree. Humans have figured out how to adapt in a world with nuclear weapons, which has shown far more risk to humanity than all but the most outlandish predictions of AI risk. You haven't presented one piece of evidence that says humans won't figure out how to adapt, or are incapable, unlikely, or unwilling to do so, whereafter humanity ceases to exist.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#128

Post by pixl97 »

stale2002 wrote: post #27
jacquesm wrote: post #18

Earlier quoted context omitted.

> when we're (say) half way there, the amount of societal upheaval will be enormous. I had a conversation with a C level exec of a large company last week around this theme. My suggestion that limited AI such as self driving cars has the potential to create a vast number of extremely frustrated individuals making a second round of 'Sabotage' and Luddites a definite possibility was waved away as if those people don't…

If self driving cars/trucks become a thing, and cause millions of people to be unemployed, there is ALSO a correspondingly massive economy increase. IE, the world is now massively richer because of all this awesome new technology. Yes, there could be some short term disruption, but honestly I think things will end up fine, just because of the massive amount of extra money and wealth that the world will have that coul…

You seem to forget why the luddites were luddities in the first place.

Yes, fabric was much cheaper, but in the meantime they were starving to death. The groups that make billions off this technology will lobby to keep more of their earnings so this vast wealth redistribution you want to happen, wont.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#129

Post by j2kun »

peferron wrote: post #119
j2kun wrote: post #104

Earlier quoted context omitted.

That would be at the top of my list. And then there's nuclear attacks/disaster, asteroid hitting the earth, superbugs and biological weapons gone awry, physics experiment gone awry creating a black hole... I put superintelligent AI down towards alien invasion on my personal list of humanity-ending risks.

We rank all these scenarios very differently then :) Climate change and nuclear war are huge issues, and we correctly invest orders of magnitude more money and time (which might still not be enough) trying to prevent or limit these than trying to prevent the birth of a malicious AGI, but they are extremely unlikely to lead to a complete extinction of the human race. Regarding the other scenarios, if you allow me to m…

Musk says AI is the single biggest existential threat (unless I'm mistaken and someone else similarly famous said that, maybe Thiel?). From this it appears that you, along with the rest of society (as you mentioned by how much is spent on oversight), think he is wrong.

Nobody is arguing against funding AI. It's the fear-mongering we disagree with. It harms the field.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#130

Post by PeterisP »

miguelrochefort wrote: post #37

Regarding "killer robots": If we can't make "paperclip maximization" the main goal of a some human, why do we expect to be able to make it the main goal of some AGI?

We can't make anything the main goal of some human, because for us us (just like any other animal) the goal system is genetically hardwired to a bunch of complex feedback loops (hunger, sexual drive, pain, social status feedback, etc etc etc), and we can't turn off any of them, much less all of them, without fundamental alterations of something that's formed very early in evolution of multicellular life.

Any AGI system will also have some set of goals, just like we do. For us the goal is to (vaguely speaking) "feel good" and all the direct things that affect that, including feelings caused by anticipation of future events, etc - but this set of goals is pretty much arbitrary if these goals didn't have to survive through millions of years of natural selection and orthogonal to intelligence power.

I mean, if it doesn't have goals, then it won't do anything, it's not an agent. You could have very smart narrow systems that aren't "agentive" and just e.g. provide answers to very complex questions, but whenever we talk about general artificial intelligence we are talking about a system that has a feedback loop with the external world, i.e., it does stuff, observes the results, learns from that (i.e. is self-modifying in some sense) and decides on further actions - which means having some goals.

Post reply on HN