Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introduction to Reinforcement Learning (2015)

deepmind.com

Post reply on HN
Print view View original

11–20 of 45 posts

Re: Introduction to Reinforcement Learning (2015)

#11

Post by WanderPanda »

bonoboTP wrote: post #10
jsemrau wrote: post #8

Earlier quoted context omitted.

When the first PC with Basic launched in the 80s many people wanted to develop for it. When the iPhone Appstore launched, many people started to build apps in the ecosystem. While it might be it bit too early to compare RL to those advances in technology. I personally feel there is huge potential. I might be wrong though. And I am fine with that.

RL isn't new though, the foundational results are about 25 years old.

And it feels a bit like it is stalling (at least in continuous control)

Re: Introduction to Reinforcement Learning (2015)

#12

Post by dmarchand90 »

bonoboTP wrote: post #10
jsemrau wrote: post #8

Earlier quoted context omitted.

When the first PC with Basic launched in the 80s many people wanted to develop for it. When the iPhone Appstore launched, many people started to build apps in the ecosystem. While it might be it bit too early to compare RL to those advances in technology. I personally feel there is huge potential. I might be wrong though. And I am fine with that.

RL isn't new though, the foundational results are about 25 years old.

That's still very analogous to the first PCs. By that point there had been decades of foundational computer work

Re: Introduction to Reinforcement Learning (2015)

#14

Post by bitL »

jsemrau wrote: post #8
vojta_letal wrote: post #6

Earlier quoted context omitted.

Does world really work like that?

When the first PC with Basic launched in the 80s many people wanted to develop for it. When the iPhone Appstore launched, many people started to build apps in the ecosystem. While it might be it bit too early to compare RL to those advances in technology. I personally feel there is huge potential. I might be wrong though. And I am fine with that.

RL needs a supercomputer and its code is usually too fragile - making a trivial mistake anywhere (missing a constant multiplication, swapping the order of two consecutive lines of code etc.) would likely lead to your model never converging even if you got everything else right.

Re: Introduction to Reinforcement Learning (2015)

#15

Post by Ma8ee »

bitL wrote: post #14
jsemrau wrote: post #8

Earlier quoted context omitted.

When the first PC with Basic launched in the 80s many people wanted to develop for it. When the iPhone Appstore launched, many people started to build apps in the ecosystem. While it might be it bit too early to compare RL to those advances in technology. I personally feel there is huge potential. I might be wrong though. And I am fine with that.

RL needs a supercomputer and its code is usually too fragile - making a trivial mistake anywhere (missing a constant multiplication, swapping the order of two consecutive lines of code etc.) would likely lead to your model never converging even if you got everything else right.

Which code is not fragile in that sense? I think that is a rather strange criticism.

Re: Introduction to Reinforcement Learning (2015)

#16

Post by Iv »

bitL wrote: post #14
jsemrau wrote: post #8

Earlier quoted context omitted.

When the first PC with Basic launched in the 80s many people wanted to develop for it. When the iPhone Appstore launched, many people started to build apps in the ecosystem. While it might be it bit too early to compare RL to those advances in technology. I personally feel there is huge potential. I might be wrong though. And I am fine with that.

RL needs a supercomputer and its code is usually too fragile - making a trivial mistake anywhere (missing a constant multiplication, swapping the order of two consecutive lines of code etc.) would likely lead to your model never converging even if you got everything else right.

You can do RL on an raspberry pi. Depends what problem you are trying to solve but not all of them require video analysis and billions of parameters.

Re: Introduction to Reinforcement Learning (2015)

#17

Post by cbames89 »

bitL wrote: post #14
jsemrau wrote: post #8

Earlier quoted context omitted.

When the first PC with Basic launched in the 80s many people wanted to develop for it. When the iPhone Appstore launched, many people started to build apps in the ecosystem. While it might be it bit too early to compare RL to those advances in technology. I personally feel there is huge potential. I might be wrong though. And I am fine with that.

RL needs a supercomputer and its code is usually too fragile - making a trivial mistake anywhere (missing a constant multiplication, swapping the order of two consecutive lines of code etc.) would likely lead to your model never converging even if you got everything else right.

Technical point: Value functions that are a constant multiples of each other result in the same behavior.

Re: Introduction to Reinforcement Learning (2015)

#18

Post by cbames89 »

WanderPanda wrote: post #11
bonoboTP wrote: post #10

Earlier quoted context omitted.

RL isn't new though, the foundational results are about 25 years old.

And it feels a bit like it is stalling (at least in continuous control)

In my opinion there's a wide open array of approaches from control that can help with this. Learning for Control is a new conference that looks at this very topic.

Re: Introduction to Reinforcement Learning (2015)

#20

Post by ilaksh »

I used to be a bit more excited about RL. I mean, it's still definitely something I have to learn, but one aspect of it _seems_ lacking to me and is messing with my motivation to learn it. I'm sure someone will happily explain all the ways I am ignorant.

It seems like there is a lot of emphasis on "direct RL" or whatever where they don't even really think about the model much, but it's I guess often inside of the policy or something?

But it seems to me as someone who has just started learning about robotics, that I absolutely need to first verify that I have an accurate model of the environment which I can inspect. It seems like a lot of RL approaches might not even be able to supply that.

I mean what I am stuck on as far as creating a robot (or virtual robot) is having a vision system that does all of the hard things I want. I feel like if I can detect edges and surfaces and shapes in 3D, parts of objects and objects, with orientation etc., and in a way I can display and manipulate it, that level of understanding will give me a firm base to build the rest of learning and planning on.

I know all of that is very hard. It seems like they must have tried that for awhile and then kind of gave up to head down the current direction of RL? Or just decided it wasn't important. I still think it's important.

Post reply on HN