TensorForce: A TensorFlow library for applied reinforcement learning
1–9 of 9 posts
Re: TensorForce: A TensorFlow library for applied reinforcement learning
#2In the past, i've found that these higher level libraries built on top of TF are useful for quick model building, but should be used cautiously. By having default hyperparameters it can be easy to blindly build semi-working models without knowing what's happening. I would recommend either reading the associated papers or implementing the models in code (at least once) before using these pre-built models. That being said, i'm really excited by this project! I think it'll save researchers a bunch of time.
Re: TensorForce: A TensorFlow library for applied reinforcement learning
#3In the past, i've found that these higher level libraries built on top of TF are useful for quick model building, but should be used cautiously. By having default hyperparameters it can be easy to blindly build semi-working models without knowing what's happening. I would recommend either reading the associated papers or implementing the models in code (at least once) before using these pre-built models. That being s…
Disclosure: I work at Google.
This is great feedback! I'd love to hear more - if you'd like to send me some examples of what you've seen in the past with pitfalls, I'd love to share them with the team.
Thanks! aronchick (at) google
Re: TensorForce: A TensorFlow library for applied reinforcement learning
#4In the past, i've found that these higher level libraries built on top of TF are useful for quick model building, but should be used cautiously. By having default hyperparameters it can be easy to blindly build semi-working models without knowing what's happening. I would recommend either reading the associated papers or implementing the models in code (at least once) before using these pre-built models. That being s…
If I understand the project correctly it precisely does not advocate default hyperparameters but exposes all configurations through the declarative interface.
Re: TensorForce: A TensorFlow library for applied reinforcement learning
#5In the past, i've found that these higher level libraries built on top of TF are useful for quick model building, but should be used cautiously. By having default hyperparameters it can be easy to blindly build semi-working models without knowing what's happening. I would recommend either reading the associated papers or implementing the models in code (at least once) before using these pre-built models. That being s…
If I understand the project correctly it precisely does not advocate default hyperparameters but exposes all configurations through the declarative interface.
I may have used the term hyperparameter too loosely. Yes, this project does a good job on taking a configuration first approach, but even they set some defaults. For example, they set relu as their default layer activation function. I haven't had time to see what other such defaults are being set.
Re: TensorForce: A TensorFlow library for applied reinforcement learning
#6In the past, i've found that these higher level libraries built on top of TF are useful for quick model building, but should be used cautiously. By having default hyperparameters it can be easy to blindly build semi-working models without knowing what's happening. I would recommend either reading the associated papers or implementing the models in code (at least once) before using these pre-built models. That being s…
I like their focus on flexibility. I've tried a few deep RL implementations in the past and run into issues like their DQN or A3C implementation being hardwired in a number of ways to working only on ALE, with no way to use it on other problems (eg the CNN dimensions are hardwired).
Re: TensorForce: A TensorFlow library for applied reinforcement learning
#7Earlier quoted context omitted.
If I understand the project correctly it precisely does not advocate default hyperparameters but exposes all configurations through the declarative interface.
I may have used the term hyperparameter too loosely. Yes, this project does a good job on taking a configuration first approach, but even they set some defaults. For example, they set relu as their default layer activation function. I haven't had time to see what other such defaults are being set.
Thanks for your feedback! To clarify our philosophy regarding default configurations: On the one hand, we try to make all hyperparameters and settings of the agents/models centrally configurable, by specifying one configuration object/file. On the other hand, we try to provide a set of default values, where it makes sense for the applied user, for whom the full range of hyperparameters is probably not interesting. In doing so, we try to combine "the best of both worlds", but sometimes that might lead to conflicts. This is something we're actively working on, to get the balance right (and comments are very welcome, best on GitHub).
Re: TensorForce: A TensorFlow library for applied reinforcement learning
#8This is super cool. Might be replacing something I've been building with it.
Are there any benchmarks I could check out? Especially for a3c.
Re: TensorForce: A TensorFlow library for applied reinforcement learning
#9This is super cool. Might be replacing something I've been building with it. Are there any benchmarks I could check out? Especially for a3c.
Hey, glad you like it! We haven't properly benchmarked our models yet, but it's one of the things at the top of our ToDo list. (However, we will only run benchmarks on some selected "standard" environments, given our limited resources.)