It seems to me that they are basically describing a variational formulation of the "optimization perspective" of reinforcement learning, which is cool, but I am confused... where is the supervised learning? Like what is the input and what is the output?
The way I understand it, the two subproblems are supervised in the sense that they are trained using data sampled from a fixed distribution, instead of data sampled from a distribution that changes as you update your model, as it is usually the case in RL. This makes the training more stable.
Actually supervised learning is "learning missing data dimension" by parameter tuning via associative learning rules.
Gradient descents are a special case of associative learning rules assuming all data points the same importance.
A type of associative learning rules is Hebbian learning rule.
In the very fundamental we only need associative learning. Of course, for practical application we need diverse tools with different conceptual frameworks to choose for commercial: human resource or cost performance.
> I'm also wondering why the authors didn't publish any experiments to show that it works... This is a blog post. It cites three of the authors' papers that each contain empirical results. The abstract of the first ends: "We formally show that this iterated supervised learning procedure optimizes a bound on the RL objective, derive performance bounds of the learned policy, and empirically demonstrate improved goal-re…
> empirically demonstrate improved goal-reaching performance and robustness over current RL algorithms It's interesting that their choice of current algorithms includes PPO but not e.g. Deepmind's Rainbow agent that achieved state of the art performance on many measures: https://arxiv.org/abs/1710.02298
They mention Rainbow in the related work section of the third paper listed there, Kumar, A., Peng, X. B., & Levine, S. (2019). Reward-Conditioned Policies. arXiv:1912.13465 as part of this remark: "they are also known to be notoriously challenging to use effectively, due to sensitivity to hyper parameters, high sample complexity, and a range of important and delicate implementation choices that have a large effect on performance [5, 6, 12, 15, 23, 24, 46]."
Actually supervised learning is "learning missing data dimension" by parameter tuning via associative learning rules. Gradient descents are a special case of associative learning rules assuming all data points the same importance. A type of associative learning rules is Hebbian learning rule. In the very fundamental we only need associative learning. Of course, for practical application we need diverse tools with dif…
I'd like to see a resurgance in pattern mining and association rule mining. That stuff is do awesome and useful!
Actually supervised learning is "learning missing data dimension" by parameter tuning via associative learning rules. Gradient descents are a special case of associative learning rules assuming all data points the same importance. A type of associative learning rules is Hebbian learning rule. In the very fundamental we only need associative learning. Of course, for practical application we need diverse tools with dif…
> Gradient descents are a special case of associative learning rules assuming all data points the same importance.
No. Gradient descent is an optimization method that has nothing to do with learning. It is used to optimize parameterized functions that are said to be "learning", but it's not the only approach. It is also trivially easy, and not uncommon, to use different weights for different data points.
Can you clarify what you mean with associative learning?
It seems to me that they are basically describing a variational formulation of the "optimization perspective" of reinforcement learning, which is cool, but I am confused... where is the supervised learning? Like what is the input and what is the output?
The way I understand it, the two subproblems are supervised in the sense that they are trained using data sampled from a fixed distribution, instead of data sampled from a distribution that changes as you update your model, as it is usually the case in RL. This makes the training more stable.
It seems to me that they are basically describing a variational formulation of the "optimization perspective" of reinforcement learning, which is cool, but I am confused... where is the supervised learning? Like what is the input and what is the output?
It seems more as if the authors are abusing terms from Machine Learning like "Supervised Learning".
It seems to me that they are basically describing a variational formulation of the "optimization perspective" of reinforcement learning, which is cool, but I am confused... where is the supervised learning? Like what is the input and what is the output?
It seems more as if the authors are abusing terms from Machine Learning like "Supervised Learning".