This repository is a fork of the Nathan Sprague implementation of the deep Q-learning algorithm described in:

Playing Atari with Deep Reinforcement Learning Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, Martin Riedmiller

and

Mnih, Volodymyr, et al. "Human-level control through deep reinforcement learning." Nature 518.7540 (2015): 529-533.

We use the DQN algorithm to learn the strategies for Atari games from the RAM state of the machine.

• A reasonably modern NVIDIA GPU
• OpenCV
• Theano (https://github.com/Theano/Theano)
• Lasagne (https://github.com/Lasagne/Lasagne
• Pylearn2 (https://github.com/lisa-lab/pylearn2)
• Arcade Learning Environment (https://github.com/sygi/Arcade-Learning-Environment (we use our fork of ALE which draws state of the RAM))

The script dep_script.sh can be used to install all dependencies under Ubuntu.

TODO: describe all the experiments and how to run the code.

• https://github.com/spragunr/deep_q_rl

  Original Nathan Sprague implementation of DQN.

• https://sites.google.com/a/deepmind.com/dqn

  This is the code DeepMind used for the Nature paper. The license only permits the code to be used for "evaluating and reviewing" the claims made in the paper.

• https://github.com/muupan/dqn-in-the-caffe

  Working Caffe-based implementation. (I haven't tried it, but there is a video of the agent playing Pong successfully.)

• https://github.com/kristjankorjus/Replicating-DeepMind

  Defunct? As far as I know, this package was never fully functional. The project is described here: http://robohub.org/artificial-general-intelligence-that-plays-atari-video-games-how-did-deepmind-do-it/

• https://github.com/brian473/neural_rl

  This is an almost-working implementation developed during Spring 2014 by my student Brian Brown. I haven't reused his code, but Brian and I worked together to puzzle through some of the blank areas of the original paper.