+0
日增长
5,546
stars
1,337
forks
190
watchers
158
prs
241
issues
8
releases
README
Deep Reinforcement Learning for Keras
What is it?
keras-rl implements some state-of-the art deep reinforcement learning algorithms in Python and seamlessly integrates with the deep learning library Keras.
Furthermore, keras-rl works with OpenAI Gym out of the box. This means that evaluating and playing around with different algorithms is easy.
Of course you can extend keras-rl according to your own needs. You can use built-in Keras callbacks and metrics or define your own.
Even more so, it is easy to implement your own environments and even algorithms by simply extending some simple abstract classes. Documentation is available online.
What is included?
As of today, the following algorithms have been implemented:
- [x] Deep Q Learning (DQN) [[1]](http://arxiv.org/abs/1312.5602), [[2]](https://www.nature.com/articles/nature14236)
- [x] Double DQN [[3]](http://arxiv.org/abs/1509.06461)
- [x] Deep Deterministic Policy Gradient (DDPG) [[4]](http://arxiv.org/abs/1509.02971)
- [x] Continuous DQN (CDQN or NAF) [[6]](http://arxiv.org/abs/1603.00748)
- [x] Cross-Entropy Method (CEM) [[7]](http://learning.mpi-sws.org/mlss2016/slides/2016-MLSS-RL.pdf), [[8]](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.81.6579&rep=rep1&type=pdf)
- [x] Dueling network DQN (Dueling DQN) [[9]](https://arxiv.org/abs/1511.06581)
- [x] Deep SARSA [[10]](http://people.inf.elte.hu/lorincz/Files/RL_2006/SuttonBook.pdf)
- [ ] Asynchronous Advantage Actor-Critic (A3C) [[5]](http://arxiv.org/abs/1602.01783)
- [ ] Proximal Policy Optimization Algorithms (PPO) [[11]](https://arxiv.org/abs/1707.06347)
You can find more information on each agent in the doc.
Installation
- Install Keras-RL from Pypi (recommended):
pip install keras-rl
- Install from Github source:
git clone https://github.com/keras-rl/keras-rl.git
cd keras-rl
python setup.py install
Examples
If you want to run the examples, you'll also have to install:
- gym by OpenAI: Installation instruction
- h5py: simply run pip install h5py
For atari example you will also need:
- Pillow: pip install Pillow
- gym[atari]: Atari module for gym. Use pip install gym[atari]
Once you have installed everything, you can try out a simple example:
bash
python examples/dqn_cartpole.py
This is a very simple example and it should converge relatively quickly, so it's a great way to get started!
It also visualizes the game during training, so you can watch it learn. How cool is that?
Some sample weights are available on keras-rl-weights.
If you have questions or problems, please file an issue or, even better, fix the problem yourself and submit a pull request!
External Projects
- Starcraft II Learning Environment
You're using Keras-RL on a project? Open a PR and share it!
Visualizing Training Metrics
To see graphs of your training progress and compare across runs, run pip install wandb and add the WandbLogger callback to your agent's fit() call:
python
from rl.callbacks imTop issues
#278Policy gradient agent - no intention to add?7 comments · 2019-01-02#287DDPG with MultiInputProcessor not working6 comments · 2019-01-17#380Issue importing keras-rl on tensorflow-macos5 comments · 2021-09-10#397Recommendation to synchronize tf, keras and keras rl versions?3 comments · 2024-03-27#285DDPG on Discrete Action Space3 comments · 2019-01-14#388Value error when running DQN.fit2 comments · 2022-04-16#399AttributeError: Tensor.op is meaningless when eager execution is enabled.1 comments · 2024-07-11#386How to set the "nb_steps_warmup" and "nb_steps" properly?1 comments · 2022-04-04
Latest release
v0.4.2 v0.4.2 2018-05-01- Fix Tensorboard support - Improve documentation