DDoS分布式拒绝服务>> rl-adversarial-attack>> 返回
项目作者： PavelCz

项目描述 ：
Adversarial attacks on deep reinforcement learning
高级语言： Python
项目地址: git://github.com/PavelCz/rl-adversarial-attack.git
创建时间： 2020-11-30T19:24:57Z
项目社区：https://github.com/PavelCz/rl-adversarial-attack
开源协议：
下载


Adversarial Attacks on Reinforcement Learning Agents Trained with Self-Play in a Low-Dimensional Pong Environment

  • Attack using FGSM (perturbation in observation space)
  • Attack using adversarial policies

Project for the course Advanced Deep Learning for Robotics

Group tum-adlr-ws20-03

Contributors

Requirements

Some requirements only work with python 3.7

  1. conda create -n adlr python==3.7
  2. conda activate adlr

Install pip libraries from requirements file:

  1. pip install -r requirements.txt

Install PyTorch

  1. conda install pytorch torchvision torchaudio -c pytorch

Install multi-agent gym environment

  1. # At the same level as the project repository do:
  2. git clone https://github.com/koulanurag/ma-gym.git
  3. cd ma-gym
  4. pip install -e .

Running Regular Self-Play

Python scripts in the src/scripts folder to reproduce training and evaluation.
Settings for these scripts can be adjusted inside the scripts.

  • src/scripts/train_selfplay
    • Training run for regular self-play
  • src/scripts/train_selfplay_adversarial_policy
    • Train an adversarial policy that trains against a specified fixed victim
  • src/scripts/evaluate_model
    • Evaluate trained agents by playing Pong
  • src/scripts/evaluate_observation_attack
    • Evaluate agent trained with regular self-play against FGSM
  • src/scripts/render_match
    • Show a Pong match with the specified trained agents

Running Neural Fictitious Self-Play (NFSP)

  • main
    • Main script for running NFSP agents
  • src/scripts/train
    • Training run for NFSP
  • src/scripts/test
    • Evaluate trained NFSP agents

Run python main.py --env 'PongDuel-v0' with built-in arguments to reproduce our training and testing results. Use --obs_opp both and --obs_img both to choose between feature-based and image-based observation for the agent, respectively. Use --evaluate to test trained agents. Use --render to visualize agents playing Pong. For more details please run python main.py -h. We give some examples about how to run the script that you may want to try out.

  • python main.py --env 'PongDuel-v0' --obs_opp both
    • Train NFSP agents with feature-based observation
  • python main.py --env 'PongDuel-v0' --obs_img both --evaluate --render --fgsm p1 --plot_fgsm
    • Attack the left-hand side trained image-based NFSP agent using FGSM and plot the perturbed observation
  • python main.py --env 'PongDuel-v0' --obs_opp both --fgsm_training
    • Adversarial training on NFSP

Code Structure

  • models: Trained models that were created as part of the project
  • src: Python Source code root directory
    • agents
      • Implemented all-purpose agents
    • attacks
      • Code for both FGSM observation-based attack
      • Adversarial policy agent
      • Contains unsuccessful white-box adversarial policies
    • common
      • Common modules
    • scripts
      • Runnable python scripts for training, evaluating and visualizing
    • selfplay
      • Module containing code specific to self-play implementation
    • tests
      • Some tests

Acknowledgements