I similarly did not get in to using MuJoCo because I want all my work to be reproducible with open source. So I spent time trying to get RL working in Godot video game engine, but ran in to some road blocks. I know MuJoCo already has all the necessary pipelines so this is exciting indeed!
Surely that kind of use would have been fine. What they're trying to do with that questionnaire must be to determine if you're making money from the venture. You can share your diary with others and still call it personal, after all. (Once you get a publisher involved, not so much.)
you could install that license on one physical machine. so say you want to use a Colab or cloud GPU or something, or your university compute cluster, it wouldn’t work.
I tend to be more than a little skeptical of organizations like DeepMind and (especially) OpenAI lately, when it comes to their pledges to democratize their advances in AI/ML/robotics. This, however, is 100% a great move by DeepMind in the support of the community as a whole. Kudos to the team, and a big thanks as well.
Great news! It was a real issue that so much RL research relied on non-free software. It was a problem that a big pile of money could solve, so I'm glad someone with a big pile of money did :)
as an undergrad in the 90s I invested thousands of dollars of my own money to work on machine learning. Worth every penny, even if I had to miss out on some cool parties.
I'm not even sure what virtue you're trying to signal here. Just because someone used the limited funds they had available to obtain the software and educational resources they were interested in, they've exercised some sort of inappropriate privilege?
Isn't that the whole idea behind college? Spend money to learn cool stuff?
Ok, I'll spell it out. Presenting the idea that MuJoCo having a license fee rather than being free is better, not worse, or even that the difference doesn't matter because "well, I paid for stuff" is the inappropriate part of the privilege. Given a choice between "I can learn X but I need to spend significant money on it" and "I can learn X for free", with everything else equal arguing for the former is exclusionary.
with everything else equal arguing for the former is exclusionary.
Sure. But who argued for that? They merely pointed out that they considered this particular physics engine worth spending money on. The implication was that because money is typically limited for a college student, the software must really be worthwhile. The implication was not "LOL, you must be broke and can't afford good software."
The implication j was responding to was "if you really cared, a mere few hundred dollars wouldn't matter because you'd be able to find the money." If you don't read that into the message chain, fine, but I do.
Well, if you're working to pay your rent, bills, and food expenses, and you're only working part time (because you're a full time undergraduate student), you may not have a lot left over to pay for software licenses.
which is precisely the reason I went with linux: so I could maximize my hardware resources while eliminating license costs. By not buying whatever MS compiler in those days, I was able to max out my workstation with 32MB RAM.
Working in undergrad and having thousands left over after paying for food and a place to sleep certainly can't be assumed to be. That's where my pay went.
I was very fortunate to not have to pay tuition or board, but if I had to, I wouldn't have spent my money to pay for that (I would have taken out loans) and still spent my money on RAM.
But they can accept that the world is not geared towards amateur dabblers, and that a few hundreds of $ for a license (a license, meaning to legalize production usage, you can always copy from somewhere to "dabble") is nothing.
The CS world has certainly been geared towards the hobbyist budget for some time now. Would be good to keep it that way, even if it meant sacrificing a supposed 1% extra accuracy in the latest "top conference" (sure if their sponsors are the companies publishing huge models it won't happen even if it should).
the price was reasonable for a good piece of software, but for the way people want to do ML research now (launch tons of parallel jobs on lots of cloud machines) the price, and more importantly, the burden of managing licenses, was prohibitive.
There was a free license for individual users, with the non-free license applicable to those who were receiving financial support (i.e. academic / industry researchers, etc.). The individual license was very popular and heavily utilized.
This is huge for the open source ML community because so many academic and commercial projects rely on MuJoCo which is out of reach for folks working on hobby projects or at tiny startups. This makes a lot more AI environments work for everyone. Major props to DeepMind
I used to use a rattleback as a test back when I was working on accurate but slow physics engines in the 1990s. That's an ellipsoid cut by a plane that isn't axis-aligned. You spin it, and it spins, stops, rocks, and spins a bit in the other direction. Very similar to what they're doing.
The key to this is a multi-point contact model. If your contacts are single points, the frictional effects are wrong. It's not that hard to do this. I held a patent on that (#6,067,096), and there was a product called Falling Bodies, the first ragdoll system that really worked.
It was just too early in the 1990s. 100 MIPS was not enough.
Google Research already maintains the Bullet Engine as well (although it mostly seems to be Erwin Coumans maintaing it)
Each have their own strengths, there is no "definitive" physics engine. Bullet is probably the most generalist engine, but it suffers a bit because of this approach (it's split between the old 2.0 C++ API vs. the new PyBullet API with vastly different functionalities)
MuJoCo going open source is good for the open source community. PyBullet is build on top of a plain C API (wrapping C++ Bullet API).
Note that there is also Brax and Tiny Differentiable Simulator, developed within Google.
One strong advantage of MuJoCo is its plain vanilla C bindings. Hardwiring an API like this to C++ or Python makes life much harder than necessary for users of other languages. Conversely, it's very easy to wrap a well-thought-out C API in your language of choice.
Super impressed with the data structures (and documentation) they've already released.
It seems great, but the simulation geometry options feel bit limited? What would be really great would be the ability to simulate scenes with objects imported from STEP/IGES files, so you can do more of the modeling in actual CAD tools. Instead, MuJoCo seems more oriented towards using its own primitives, not sure how that works out with more complex situations.
If you can turn it into a collection of STL or MSH files then you can simulate it in MuJoCo. There are some caveats - MuJoCo will use the convex hull of the mesh for collisions, so if you want to accurately simulate collisions with concave features then you'll need to split your mesh into convex submeshes (there are various third party tools that can automate this). Sometimes you need to tweak the mesh geometry a bit in order to get the right contact behaviour, and in some cases it's easier to use primitives for key collision geometry and just use the meshes for visualisation.
They're not in the file-reading business. There's a staggering variety of 3D object file formats, both open and proprietary, so it's logical enough to let the application worry about reading the geometric data and converting it to the representation needed for physics simulation.
Nope. PhysX is all about realtime and gaming is the end use it was designed around. TFA has a section about why that’s bad. The different end uses mean there are miles of differences between them even though they’re both physics simulators.
A lot of RL problems can be adequately simulated in either Mujoco or PhysX. So even though as you pointed out they have key differences, there is a pretty big overlap in use cases. So in that sense, I believe it still does qualify as competition.
I have personally spent tons of time dealing with models exploiting physics issues. It’s a major issue. It’s true that either can adequately simulate the setups, but is it true that either can be equally exploited? I genuinely don’t know. I kinda assumed physx is more exploitable.
I agree with you that PhysX is definitely easier to exploit, and you will lose time because of that.
But on the other hand, for many problems, the gain in training time using PhysX is just so massive (2 or 3 orders of magnitude), that you can then train with a much more aggressive sim2real policy to mitigate that to some extent, or even spend time debugging manually and still ending up more productive.
I think this is referring to inaccuracies of simulation which the RL model can learn to exploit. Sort of a local minimum that doesn't exist in the real world.
Say you're teaching a robot to walk, but the joint constraints in the simulator aren't quite right. It would be possible for the policy to exert more torque on the joint than it's supposed to be able to in real life. Take that policy and put it on a real robot and it won't work.
The docs talk about that a little [1], with the answer boiling down to "Yes, the necessary hooks are there, but no, you don't want to use this particular engine for fluid dynamics."
The docs are definitely a strong point, nicely written and presented.
When I was learning Deep RL, I edged on what could be done through OpenAI gym, or self-made environments, I decided to begin working with MuJoCo.
But for an undergrad student from a developing country, the fees were extremely high.
So, I just didn't go forward with Deep RL much. I dabbled for a while and tried using PyBullet. Did not go that well.
Also thought about creating custom environments in Unity and then apply Deep RL in there. But it took too much effort.
Creating a complex enough environment and getting used to it were, in themselves- full projects that took long term commitment.
As a result, I did not learn much Deep RL to be able to apply it to complex, real-world-like scenarios.
I think that it is going to change.
I am excited about books, blogs, and videos that are going to be created as MuJoCo is open-sourced and available for free.
I, personally, will get back into Deep RL the next opportunity I get.