倒排索引/搜索引擎>> LWDRLD>> 返回
项目作者： LQNew

项目描述 ：
Lightweight deep RL Libraray for discrete control.
高级语言： Python
项目地址: git://github.com/LQNew/LWDRLD.git
创建时间： 2021-04-22T03:29:34Z
项目社区：https://github.com/LQNew/LWDRLD
开源协议：MIT License
下载


LWDRLD: LightWeight Deep Reinforcement Learning libraray for Discrete control

Modularized implementation of popular deep RL algorithms in PyTorch. LWDRLD is a deep reinforcement learning (RL) library which is inspired by some other deep RL code bases (i.e., Kchu/DeepRL_PyTorch, openai/baselines, and ShangtongZhang/DeepRL).

These algorithms will make it easier for the research community to replicate, refine, and identify new ideas. Our implementation of state-of-the-art algorithms is roughly on par with the scores in published papers.

Supported algorithms

algorithm Discrete Control Value-based Policy-based
Deep Q-Learning (DQN) :white_check_mark: :white_check_mark: :no_entry:
Double DQN :white_check_mark: :white_check_mark: :no_entry:
Dueling DQN :white_check_mark: :white_check_mark: :no_entry:
Prioritized Experience Replay (PER) :white_check_mark: :white_check_mark: :no_entry:

TODO: Waiting to be implemented…

  • N-Step DQN
  • Proximal Policy Optimization (PPO) for Discrete Control
  • Advantage Actor-Critic (A2C)
  • Discrete Soft Actor-Critic (SAC)

Instructions

Recommend: Run with Docker

  1. # python        3.6    (apt)
  2. # pytorch       1.4.0  (pip)
  3. # tensorflow    1.14.0 (pip)
  4. # DMC Control Suite
  5. # MuJoCo
  6. # Atari
  7. # Attention: Our dockerfile needs `mjkey.txt`!! If you just want to run the Atari env, you can change `Dockerfile` and make your customization.
  8. cd dockerfiles
  9. docker build . -t lwdrld

For other dockerfiles, you can go to RL Dockefiles.

Launch experiments

  1. # eg.
  2. # ============== DQN
  3. CUDA_VISIBLE_DEVICES=0 python main.py --policy "DQN" --env "Pong" --seed 0 --exp_name DQN-PongNoFrameskip-v4  # env_name: PongNoFrameskip-v4, algorithm: DQN, CUDA_Num : 0, seed: 0
  4. # ============== Double DQN
  5. CUDA_VISIBLE_DEVICES=0 python main.py --policy "Double_DQN" --env "Pong" --seed 0 --exp_name Double_DQN-PongNoFrameskip-v4
  6. # ============== Dueling DQN
  7. CUDA_VISIBLE_DEVICES=0 python main.py --policy "Dueling_DQN" --env "Breakout" --seed 0 --exp_name Dueling_DQN-BreakoutNoFrameskip-v4
  8. # ============== Dueling Double DQN
  9. CUDA_VISIBLE_DEVICES=0 python main.py --policy "Dueling_Double_DQN" --env "Pong" --seed 0 --exp_name Dueling_Double_DQN-PongNoFrameskip-v4
  10. # ============== DQN + PER
  11. CUDA_VISIBLE_DEVICES=0 python main_per.py --policy "DQN_per" --env "Atlantis" --seed 0 --exp_name DQN_per-AtlantisNoFrameskip-v4
  12. # ============== Double DQN + PER
  13. CUDA_VISIBLE_DEVICES=0 python main_per.py --policy "Double_DQN_per" --env "Pong" --seed 0 --exp_name Double_DQN_per-PongNoFrameskip-v4
  14. # ============== Dueling DQN + PER
  15. CUDA_VISIBLE_DEVICES=0 python main_per.py --policy "Dueling_DQN_per" --env "Alien" --seed 0 --exp_name Dueling_DQN_per-AlienNoFrameskip-v4
  16. # ============== Dueling Double DQN + PER
  17. CUDA_VISIBLE_DEVICES=0 python main_per.py --policy "Dueling_Double_DQN_per" --env "Atlantis" --seed 0 --exp_name Dueling_Double_DQN_per-AtlantisNoFrameskip-v4

Plot results

  1. # eg. Notice: `-l` denotes labels, `data/DQN-PongNoFrameskip-v4/` represents the collecting dataset, 
  2. # `-s` represents smoothing value, `-t` represents title.
  3. python spinupUtils/plot.py \
  4.     data/DQN-PongNoFrameskip-v4/ \
  5.     -l DQN -s 10 -t PongNoFrameskip-v4

Performance on Atari



Citation

  1. @misc{QingLi2021lwdrld,
  2.   author = {Qing Li},
  3.   title = {LWDRLD: LightWeight Deep Reinforcement Learning libraray for Discrete control},
  4.   year = {2021},
  5.   publisher = {GitHub},
  6.   journal = {GitHub repository},
  7.   howpublished = {\url{https://github.com/LQNew/LWDRLD}}
  8. }