21–30 of 77 posts
This is really good news. It was a really contentious issue in RL and robotics research to be so reliant on proprietary (and very expensive) software.
Earlier quoted context omitted.
A lot of RL problems can be adequately simulated in either Mujoco or PhysX. So even though as you pointed out they have key differences, there is a pretty big overlap in use cases. So in that sense, I believe it still does qualify as competition.
I have personally spent tons of time dealing with models exploiting physics issues. It’s a major issue. It’s true that either can adequately simulate the setups, but is it true that either can be equally exploited? I genuinely don’t know. I kinda assumed physx is more exploitable.
But on the other hand, for many problems, the gain in training time using PhysX is just so massive (2 or 3 orders of magnitude), that you can then train with a much more aggressive sim2real policy to mitigate that to some extent, or even spend time debugging manually and still ending up more productive.
Earlier quoted context omitted.
For undergrad students trying to dabble in reinforcement learning, several hundreds of dollars can be quite a lot.
But they can accept that the world is not geared towards amateur dabblers, and that a few hundreds of $ for a license (a license, meaning to legalize production usage, you can always copy from somewhere to "dabble") is nothing.
Earlier quoted context omitted.
A lot of RL problems can be adequately simulated in either Mujoco or PhysX. So even though as you pointed out they have key differences, there is a pretty big overlap in use cases. So in that sense, I believe it still does qualify as competition.
I have personally spent tons of time dealing with models exploiting physics issues. It’s a major issue. It’s true that either can adequately simulate the setups, but is it true that either can be equally exploited? I genuinely don’t know. I kinda assumed physx is more exploitable.
I tend to be more than a little skeptical of organizations like DeepMind and (especially) OpenAI lately, when it comes to their pledges to democratize their advances in AI/ML/robotics. This, however, is 100% a great move by DeepMind in the support of the community as a whole. Kudos to the team, and a big thanks as well.
Earlier quoted context omitted.
as an undergrad in the 90s I invested thousands of dollars of my own money to work on machine learning. Worth every penny, even if I had to miss out on some cool parties.
Wonderful that you had thousands of dollars as an undergrad to be able to do so.
When I was learning Deep RL, I edged on what could be done through OpenAI gym, or self-made environments, I decided to begin working with MuJoCo.
But for an undergrad student from a developing country, the fees were extremely high.
So, I just didn't go forward with Deep RL much. I dabbled for a while and tried using PyBullet. Did not go that well.
Also thought about creating custom environments in Unity and then apply Deep RL in there. But it took too much effort.
Creating a complex enough environment and getting used to it were, in themselves- full projects that took long term commitment.
As a result, I did not learn much Deep RL to be able to apply it to complex, real-world-like scenarios.
I think that it is going to change.
I am excited about books, blogs, and videos that are going to be created as MuJoCo is open-sourced and available for free.
I, personally, will get back into Deep RL the next opportunity I get.