I'm currently working on an RL project based on an OpenAI Gym environment and have been reviewing the different frameworks available. So far I’ve come across:
- OpenAI Baselines (more a collection of algorithms than a framework)
- Keras-RL (looked ideal but has been abandoned)
- Tensorflow Agents (An 'official'? Tensorflow library, but very basic- only one algo at present)
- rllab (Developed by OpenAI people but seems to be abandoned)
- OpenAI Lab (?)
- TensorForce
My main concerns are: 1. Soundness of the algo implementations. 2. Modularity, ease-of-use, compatibility.
I first looked at Baselines as it seemed to best address the first concern but ran into frustrations when for example the DeepQ implementation didn’t work if my Gym’s action_space was a Tuple space. I am working with a team unfamiliar with RL so want something that is as plug-n-play as possible, like Keras. So far TensorForce looks promising. Can anyone add anything more? Thanks
At least in terms of integration, TensorForce aims to be a "plug and play" library. However, RL is not at a stage right now where you can just plug an algorithm to any kind of problem and expect it to learn. Hyperparameter tuning is always necessary.
Still, TensorForce does provide pluggable implementations of state-of-the-art algorithms as well as runner utilities and environment abstractions to make it easy to connect your learning problem to it.
- OpenAI Baselines (more a collection of algorithms than a framework)
- Keras-RL (looked ideal but has been abandoned)
- Tensorflow Agents (An 'official'? Tensorflow library, but very basic- only one algo at present)
- rllab (Developed by OpenAI people but seems to be abandoned)
- OpenAI Lab (?)
- TensorForce
My main concerns are: 1. Soundness of the algo implementations. 2. Modularity, ease-of-use, compatibility.
I first looked at Baselines as it seemed to best address the first concern but ran into frustrations when for example the DeepQ implementation didn’t work if my Gym’s action_space was a Tuple space. I am working with a team unfamiliar with RL so want something that is as plug-n-play as possible, like Keras. So far TensorForce looks promising. Can anyone add anything more? Thanks