Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thoughts on OpenAI, reinforcement learning, and killer robots

fast.ai

Post reply on HN
Print view View original

91–100 of 205 posts

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#91

Post by nopinsight »

tanilama wrote: post #85
nopinsight wrote: post #78

Top people including DeepMind's CEO Demis Hassabis and Prof Stuart Russell, a AAAI and AAAS fellow who is a co-author of the AI textbook used at most of top universities, agree that AGI is definitely possible and going to happen. [1] Hassabis also stated in another session that there are probably at least half a dozen mountains to climb before reaching AGI and he would be surprised if it takes more than 20. He also s…

> Top people including DeepMind CEO Demis Hassabis and Prof Stuart Russell, a AAAI and AAAS fellow who is a co-author of the AI textbook most used at top universities, agree that AGI is definitely possible and going to happen. Even though they are high-profile people, in DL, since people still don't know a lot about it, their confidence means nothing. When I was in college, my professors/textbook alike, claimed that…

The point is whether we should be complacent and dismiss concerns just because we don't know when it will happen.

Once someone builds it, stopping it might be very difficult. Here's why: https://youtu.be/4l7Is6vOAOA (less than 9 minutes and very clearly explained).

Dismissing even a 10% chance of possible catastrophic risks is not what we practice in any other domains. Would you dismiss a concern over airplanes that have a 10% or even just a 1% chance of crashing?

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#92

Post by cshenton »

jph00 wrote: post #15
phreeza wrote: post #8

Earlier quoted context omitted.

Calling it an "Atari problem" sounds quite disparaging and misses the point. It's like calling a convolutional network doing the ImageNet task a "Doggy-detection" problem. That may be the original development problem, but the final product still helps detect cancer in CT scan images... Same goes for advances in reinforcement learning made on atari games.

Perhaps, but the jury is still very much out. The vast majority of RL applications are game playing. Very few examples of valuable applications to society or the economy. There's also plenty of evidence already that RL isn't really the right way to tackle the credit problem. E.g random search is only 10x slower.

In what circumstances is it only 10x slower? Random search is totally useless when your environment is stochastic. These algorithms aren't learning sequences of actions, in fact most use a 30 'no op' random start to avoid just that.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#93

Post by afthonos »

jacquesm wrote: post #41
_qbjt wrote: post #28

> It is hard for me to empathize with Musk’s fixation on evil super-intelligent AGI killer robots in a very distant future. I would argue this person has no business working with AI with this sort of myopic thinking. If you think "fake news" is a problem now, just wait until decentralized AI networks are able to tweet, publish articles and affect public discourse in a way that is indistinguishable from human influenc…

> I would argue this person has no business working with AI with this sort of myopic thinking. What kind of thinking would you suggest would permit someone to 'work with AI'? Should they be quaking in their boots before being allowed to work at the altar? You could argue (and with substantially more basis in fact) that Twitter and Facebook have elevated each individuals utterings to broadcast status and that even wit…

I would challenge you to power off Amazon's or Google's infrastructure. Because a dangerous AI won't live in a desktop; it will live in a geographically replicated system, across multiple power grids and legal jurisdictions—all of which is possible today. And all these systems have spent untold hours of very smart people's time to ensure they never turn off.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#94

Post by rw »

jacquesm wrote: post #67
daly wrote: post #60

I've worked in a lot of AI-related projects and was around when the AI winter arrived. These various techniques that currently work by training, either supervised or self-training, can have fatal flaws. Take, for example, some high-tech camera technology. Use it on a drone to take pictures of warships from thousands of angles. You take pictures of U.S. warships, Russian warships, and Chinese warships. You achieve 100…

That's called 'data leakage'. http://machinelearningmastery.com/data-leakage-machine-learn...

No, it's called insufficient feature engineering. Data leakage is when your test data contaminates your training data.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#95

Post by backpropaganda »

> Reinforcement learning: the study of teaching computers how to beat Atari.

This statement says more about the author and her inability to understand RL than about RL itself. RL doesn't fit fast.ai's "AI is easy" narrative therefore it's not worth doing.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#96

Post by dsacco »

_qbjt wrote: post #44
RodericDay wrote: post #35

Earlier quoted context omitted.

In fact, Chollet links a whole series of essays worth reading: 1) The Myth of a Superhuman AI ( https://www.wired.com/2017/04/the-myth-of-a-superhuman-ai/ ) 2) The Terminator Is Not Coming. ( https://www.recode.net/2015/3/2/11559576/the-terminator-is-n... ) 3) idlewords : Superintelligence - The Idea That Eats Smart People ( http://idlewords.com/talks/superintelligence.htm ) The frustrating thing is that people with…

> In fact, Chollet links a whole series of essays worth reading: Appeal to authority. You're not presenting any substantive rebuttals, just assuming it's not a concern because someone else with credentials said so. It's identical to alleging that I'm only worried because Elon Musk says so. > The frustrating thing is that people with insane amounts of resources and disposable income get swallowed up by these Basilisk-…

That's not an appeal to authority. The essays themselves are substantive; he didn't say, "Chollet says so, thus it is so."

More importantly there is an element of misuse going on here: the "fallacy fallacy" if you will. Fallacies like ad hominem and appeal to authority are very often mis-cited without substantive refutations. But in the absence of other qualifying data it is absolutely relevant to a dialectic session that you mention authorities in a subject matter. When we talk about things like consensus in sciences (like climate change, for example), that is a valid appeal to authority.

You don't have enough time or expertise to verify specialized information or even qualify its context. At some point you absolutely need to appeal to authority just to be productive. If you can assess data yourself then that's obviously better, but in the absence of that capacity the assertion of various authorities is a valid first heuristic.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#97

Post by emiliobumachar »

miguelrochefort wrote: post #37

Regarding "killer robots": If we can't make "paperclip maximization" the main goal of a some human, why do we expect to be able to make it the main goal of some AGI?

We can make "profit maximization" the main goal of several very smart humans, though that does require quite a lot of programming.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#98

Post by jph00 »

bryananderson wrote: post #86

Maybe I'm missing something here. First of all, I know a lot of the popular perception of artificial "general intelligence" is overly simplistic - I don't believe in the nerd rapture, and I agree with a lot of what was written in the three articles tweeted by Chollet that have been linked in this thread. And yet, I still don't see how AI is not a plausible existential threat. Unless you believe that some magical busi…

I agree with everything you said. However, I don't agree it's what should be hogging the money and interest right now. The impact of AI on employment (and therefore societal stability) is a much more pressing problem. As is the misuse of AI by people in many areas.

If society can get through these issues, then we may get to the point where the existential threat of superhuman AI is of most immediate concern.

Either way, we're not arguing that this issue deserves no time and attention at all. But currently it's the first (and often only) thing I'm asked about when I give talks about the future implications of AI, and it's receiving huge amounts of funding. Elon Musk, when he had the opportunity of addressing some of America's most powerful people, elected to spend his precious time discussing this issue, rather than anything else.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#99

Post by j2kun »

_qbjt wrote: post #77
jacquesm wrote: post #69

Earlier quoted context omitted.

Typically if you start making crazy claims the onus is on you to present actual arguments.

Present an argument against what? All you have done is shot down my claims with some "life isn't science fiction" garbage. You haven't posted a single comment in this thread relevant to the future implications of AI. There's nothing thought provoking about your comments. You're just being dismissive.

You are imagining super intelligent malicious AI (or AI piloted by malicious actors) with orders of magnitude more resources and reach than we have today is instantaneously implanted into today's society. That is also not relevant to the future implications of AI, because as AI evolves so will humanity. If you demand how exactly this will come about, then I will first require you to explain exactly how superintelligent AI will come about. The same answer is equally applicable and useless for both sides: "I don't know, but it definitely will."

You're also ignoring basic ideas that can defeat the apocalyptic problems you suggest. For example, "fake news" in the sense that you propose (adversaries generating synthesized videos and audio indistinguishable from reality) can be trivially defeated by cryptographic signatures. No matter how good a deep net gets, it won't beat cryptographic hardness. No chaos will ensue when major web browsers verify videos of the president signed with an official government public key, the same way they verify websites with CA's today. In fact, "fake news" today is a completely different problem, that people accept the source is legit (CNN did indeed publish an article about so and so who said X, and they did actually say X), but rather that they disagree with the meaning assigned to those facts.

You also ignore that one already has the capability to ruin a specific person's life via digital means today, without AI, just with enough dedicated time and money. Why bother with (expensive) superhuman AI targeting dissidents under an oppressive regime when instead, as most successful oppressive regimes today do, you can just lock them up or burn their house down?

In my humble opinion, if you find yourself in a situation where people don't want to present you with arguments to what you feel is logically bullet-proof reasoning, and that people are "just being dismissive," the issue is probably you, not them. Rather than bite back, the most useful approach is to reconsider your ideas. Attack them as critically as you attack the ideas of others. You may surprise yourself with what you find.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#100

Post by jph00 »

udkl wrote: post #55
jph00 wrote: post #29

Earlier quoted context omitted.

That's not how things have generally worked out in the world. Billions live in abject poverty even although the world has plenty enough to feed them and provide shelter. The issue is income distribution - and just making more money doesn't magically fix the problem.

> just making more money doesn't magically fix the problem Magically, no. With an effort yes. There are already experiments with basic pay for example. Jeremy Rifkin books such as https://www.amazon.com/End-Work-Decline-Global-Post-Market/d... & 'Zero Marginal Cost' discuss the topic.

Yes exactly. We really hope to see this effort made. It's a fixable problem, but often in history great inequality is only fixed by violent revolution, which I don't want to see happen in my (or my daughter's) time!
Post reply on HN