I did the course during my RC batch this year. Highly recommended if you're looking to learn RL.
What's an RC batch?
Introduction to Reinforcement Learning (2015)
21–30 of 45 posts
Re: Introduction to Reinforcement Learning (2015)
#22I'm taking an adaptation of this class. My professor is simply reusing Silver's slides, so I'm watching the original lecture instead. Highly recommend!
Re: Introduction to Reinforcement Learning (2015)
#23I did the course during my RC batch this year. Highly recommended if you're looking to learn RL.
What's an RC batch?
https://www.recurse.com/
>The Recurse Center is a self-directed, community-driven educational retreat for programmers based in New York City and currently operating online.
Re: Introduction to Reinforcement Learning (2015)
#24Earlier quoted context omitted.
When the first PC with Basic launched in the 80s many people wanted to develop for it. When the iPhone Appstore launched, many people started to build apps in the ecosystem. While it might be it bit too early to compare RL to those advances in technology. I personally feel there is huge potential. I might be wrong though. And I am fine with that.
RL needs a supercomputer and its code is usually too fragile - making a trivial mistake anywhere (missing a constant multiplication, swapping the order of two consecutive lines of code etc.) would likely lead to your model never converging even if you got everything else right.
The hard part of RL for the problems I've encountered in my work is that you need a simulator. Building a reliable and accurate simulator is often an immense undertaking.
Re: Introduction to Reinforcement Learning (2015)
#25That's one of these moments in life where you see tech and you know it will change the world, but don't see the problem yet.
RL is a good theoretical solution for personalization: given a user state, select an action that maximizes a long term reward (eg. revenue/engagement.) It’s tricky building the implementations because unlike Go/Chess/Atari it’s hard to simulate humans. So you have to train the agents with batches of data offline (ie. using historic data from the agent’s past actions.) This is challenging because you don’t get as many chances to try different hyper parameters. It’s starting to be used more in industry though.
Re: Introduction to Reinforcement Learning (2015)
#26That's one of these moments in life where you see tech and you know it will change the world, but don't see the problem yet.
RL is a good theoretical solution for personalization: given a user state, select an action that maximizes a long term reward (eg. revenue/engagement.) It’s tricky building the implementations because unlike Go/Chess/Atari it’s hard to simulate humans. So you have to train the agents with batches of data offline (ie. using historic data from the agent’s past actions.) This is challenging because you don’t get as many…
I’ve not kept up with the recent developments in this field - is Vowpal Wabbit widely used now? Any competitors? Or do people build their own in-house systems?
Thanks
Re: Introduction to Reinforcement Learning (2015)
#27Asking from a layman's perspective:
I've read a bit about genetic algorithms or evolutionary computation at some point. Apparently it achieves good results as it can find discrete solutions for complex, well defined problems.
Reinforcement learning is something I know even less about. But from what I gathered it is also most successful in well defined problems and systems (such as games).
So my question is: How do they relate? Is there overlap and what are the most significant conceptual differences?
Re: Introduction to Reinforcement Learning (2015)
#28Earlier quoted context omitted.
RL needs a supercomputer and its code is usually too fragile - making a trivial mistake anywhere (missing a constant multiplication, swapping the order of two consecutive lines of code etc.) would likely lead to your model never converging even if you got everything else right.
The hard part of RL for the problems I've encountered in my work is that you need a simulator. Building a reliable and accurate simulator is often an immense undertaking.
Maybe data scientists should team up (more?) with game programmers. They have a ton of experience in building very complex simulations.
Re: Introduction to Reinforcement Learning (2015)
#29On one hand you're right, methods like Q-learning are model-free and do not necessarily encode much about state dynamics. The Q-function is a feature (function) of state and while ita may not say much about the model, it does encode the most important aspect of the model in terms of solving the task. Namely, it predicts the accumulated reward conditional on next actions actions. That makes it a somewhat narrow representation of state on its own. But, if you consider an environment that has many reward signals, and you learn Q functions for each, this ensemble of Q functions can consitute a rich representation of state. Depending on what the reward functions are, the associated Q functions may be sufficient to construct a full model. so I guess my point is that the learned quantities in RL encode key aspects of state, and when you expand beyond the single task/single reward RL setting the lines between value and model can become blurred.
Re: Introduction to Reinforcement Learning (2015)
#30Hrm so are markov decision processes RL?