Live data from Hacker News

Thoughts on OpenAI, reinforcement learning, and killer robots

fast.ai

131–140 of 205 posts

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#131
post #45
post #24

Earlier quoted context omitted.

I'm working on a book for programmers who want to learn math. I can send you the first few chapters if you're interested, but I'm also interested to hear your thoughts about math in general.

I'd like to review and give feedback

I don't believe I have a way to contact you, but shoot me an email at mathintersectprogramming@gmail.com

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#132
post #60

I've worked in a lot of AI-related projects and was around when the AI winter arrived. These various techniques that currently work by training, either supervised or self-training, can have fatal flaws. Take, for example, some high-tech camera technology. Use it on a drone to take pictures of warships from thousands of angles. You take pictures of U.S. warships, Russian warships, and Chinese warships. You achieve 100…

Checkout LIME: https://github.com/marcotcr/lime presumably in this example when reviewing the test dataset you'd see the ocean light up more than the warships.

E.g. Here is an example of a wolf (ship) detection system that is actually detecting snow (area of the ocean) https://youtu.be/hUnRCxnydCc?t=58

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#133

Earlier quoted context omitted.

The point is whether we should be complacent and dismiss concerns just because we don't know when it will happen. Once someone builds it, stopping it might be very difficult. Here's why: https://youtu.be/4l7Is6vOAOA (less than 9 minutes and very clearly explained). Dismissing even a 10% chance of possible catastrophic risks is not what we practice in any other domains. Would you dismiss a concern over airplanes that…

I guess, as a theoretical computer scientist myself, my first reaction is to roll my eyes. I get that these guys are trying to drum up interest with investors/funding agencies, but "My greatest fear is that my research is too successful" is still pretty grandiose. It sounds like those physicists who thought they had figured everything out in the '30s (or AI researchers before the last winter). I think it's pretty tel…

> physicists who thought they had figured everything out in the '30s

Certain 30s physicists were making "My greatest fear is that my research is too successful" warnings that turned out to be pretty on-point.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#134
post #126

Earlier quoted context omitted.

I find RL hard and no I don't need Karpathy's or fast.ai's dumbed down version of RL. I'm talking about cutting-edge RL like PSRL or RETRACE, not policy gradient or DQN.

Well OK... I don't know why you think those are hard (let alone cutting edge - PSRL isn't new), but it seems important for you to feel like you understand things other people aren't smart enough to, so there's probably not much more I can say.

You can say RL is not hard when you manage to teach all aspects of the SOTA RL algorithm (https://arxiv.org/abs/1707.06887) to your class such that they are able to answer any question about it (not just implement it). Good luck teaching metric spaces to code monkeys.

You guys are doing good work teaching tensorflow and algorithms/models researchers are coming up with, but are slapping those same researchers by disrespecting what they're working on now. Some humility would be wise.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#135

I have the intuition that, given a specific "dangerous" goal (i.e., paperclip maximization), AGI will have too much autonomy to stick to it and AI won't have enough autonomy to make it happen. I believe that the fear of AI is unfounded.

Can you elaborate what you mean by "AGI will have too much autonomy to stick to it" ?

Any system that is capable decide that e.g. paperclip maximization is a bad goal must unavoidably have some scale of what constitutes better or worse goals... and that de facto means that whatever is at the "good" end of that scale will be the true goal of that system. But where does that scale come from?

Especially given that this scale is absolutely arbitrary, there are no competing "innate drives" (like mammals have) that would make it difficult to stick with any arbitrary goal. We're not talking about giving some intelligence orders that might conflict with what it "really wants" - we're talking about configuring the ultimate desires of that system, this configuration will define the world states that it will find more or less "desirable" given complete autonomy.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#136
post #24

Earlier quoted context omitted.

I'm working on a book for programmers who want to learn math. I can send you the first few chapters if you're interested, but I'm also interested to hear your thoughts about math in general.

I feel like I'm in the same boat: not hugely great at math (I mean I can do algebra and various geometries) but calculus and a lot of AI literature I've read that included algorithms were just very complicated for me to figure out. So I would also be interested in this book of yours. If you want my feedback I'm also available but I certainly don't expect handouts; just add me to a list to spam when you get your book…

Check your keybase folder /private/j2kun,krissiegel/ or ping me at mathintersectprogramming@gmail.com

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#137

Earlier quoted context omitted.

> Top people including DeepMind CEO Demis Hassabis and Prof Stuart Russell, a AAAI and AAAS fellow who is a co-author of the AI textbook most used at top universities, agree that AGI is definitely possible and going to happen. Even though they are high-profile people, in DL, since people still don't know a lot about it, their confidence means nothing. When I was in college, my professors/textbook alike, claimed that…

The point is whether we should be complacent and dismiss concerns just because we don't know when it will happen. Once someone builds it, stopping it might be very difficult. Here's why: https://youtu.be/4l7Is6vOAOA (less than 9 minutes and very clearly explained). Dismissing even a 10% chance of possible catastrophic risks is not what we practice in any other domains. Would you dismiss a concern over airplanes that…

Point is, nobody knows whether AGI really exists or not, based on our current approach. As I have answered in another comment section, natural language understanding haven't been cracked, at all. I mean at all.

So the reality is, it is like ancient human knows birds can fly with wings, but people can only fly in their dreams. Now, we know people can think intelligently with their brain, but no one knows how to make computer works the same. It is far too early to talk about the risk, until we have the Wright brothers of AI to enlighten us on such possibility.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#138
post #136

Earlier quoted context omitted.

I feel like I'm in the same boat: not hugely great at math (I mean I can do algebra and various geometries) but calculus and a lot of AI literature I've read that included algorithms were just very complicated for me to figure out. So I would also be interested in this book of yours. If you want my feedback I'm also available but I certainly don't expect handouts; just add me to a list to spam when you get your book…

Check your keybase folder /private/j2kun,krissiegel/ or ping me at mathintersectprogramming@gmail.com

I guess it's time I check out keybase's file stuff. Thanks!

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#139
post #126

Earlier quoted context omitted.

Well OK... I don't know why you think those are hard (let alone cutting edge - PSRL isn't new), but it seems important for you to feel like you understand things other people aren't smart enough to, so there's probably not much more I can say.

You can say RL is not hard when you manage to teach all aspects of the SOTA RL algorithm ( https://arxiv.org/abs/1707.06887 ) to your class such that they are able to answer any question about it (not just implement it). Good luck teaching metric spaces to code monkeys. You guys are doing good work teaching tensorflow and algorithms/models researchers are coming up with, but are slapping those same researchers by dis…

Not sure why the exact details of state of the art research is relevant here. Obviously that definition of RL is dumbing it down, as I'm sure Rachel knows, but the point is simple - teaching a computer to do something specific a human can without explicitly telling it whether something is good or bad, but rather have it learn on its own.

The latest research in RL isn't getting us that much closer to AGI. We can't plop a robot into the real world and tell it to use RL to learn everything.

Re: Thoughts on OpenAI, reinforcement learning, and killer robots

#140

Earlier quoted context omitted.

The point is whether we should be complacent and dismiss concerns just because we don't know when it will happen. Once someone builds it, stopping it might be very difficult. Here's why: https://youtu.be/4l7Is6vOAOA (less than 9 minutes and very clearly explained). Dismissing even a 10% chance of possible catastrophic risks is not what we practice in any other domains. Would you dismiss a concern over airplanes that…

Point is, nobody knows whether AGI really exists or not, based on our current approach. As I have answered in another comment section, natural language understanding haven't been cracked, at all. I mean at all. So the reality is, it is like ancient human knows birds can fly with wings, but people can only fly in their dreams. Now, we know people can think intelligently with their brain, but no one knows how to make c…

Unlike airplanes, AGI possesses agency and the ability to cause widespread harm and damage in today's computer-penetrated world. So if we wait until it is invented, there is no guarantee that there won't be danger or even catastrophes. Wouldn't you even agree that there is a 1% chance that AGI is possible and that it could harm us once invented?

Also, the goalpost for identifying something as a challenging cognitive skill worthy of the name AI is moved almost every time we make progress so people keep denying that we are closer to AGI. AlphaGo is the latest example.

Some AI researchers and CS people believe that Natural Language Understanding (NLU) is AI-complete, i.e. once we solve it we basically solve AI in the sense of AGI. I do not personally believe that--There are certain human cognitive skills that are not required for NLU. But I do think that solving NLU does bring us closer to solving AGI.

Let's say someone makes progress on NLU. What would be the minimum level sufficient to convince you that AGI is possible? Why minimum? Because we don't want AGI to be right at our doors before starting to prepare.

* Would getting 80% on Winograd Schema be sufficient?

Other suggestions are welcomed, including by others.

https://en.wikipedia.org/wiki/Winograd_Schema_Challenge

Post reply on HN