repository info

Author: sygi

README.md

@@ -1,6 +1,6 @@
 # Introduction 
 
-This package provides a Lasagne/Theano-based implementation of the deep
+This repository is a fork of the Nathan Sprague implementation of the deep
 Q-learning algorithm described in:
 
 [Playing Atari with Deep Reinforcement Learning](http://arxiv.org/abs/1312.5602)
@@ -11,9 +11,7 @@ and
 
 Mnih, Volodymyr, et al. "Human-level control through deep reinforcement learning." Nature 518.7540 (2015): 529-533.
 
-Here is a video showing a trained network playing breakout (using an earlier version of the code):
-
- http://youtu.be/SZ88F82KLX4
+We use the DQN algorithm to learn the strategies for Atari games from the RAM state of the machine.
 
 # Dependencies
 
@@ -22,53 +20,20 @@ Here is a video showing a trained network playing breakout (using an earlier ver
 * [Theano](http://deeplearning.net/software/theano/) ([https://github.com/Theano/Theano](https://github.com/Theano/Theano))
 * [Lasagne](http://lasagne.readthedocs.org/en/latest/) ([https://github.com/Lasagne/Lasagne](https://github.com/Lasagne/Lasagne)
 * [Pylearn2](http://deeplearning.net/software/pylearn2/) ([https://github.com/lisa-lab/pylearn2](https://github.com/lisa-lab/pylearn2))
-* [Arcade Learning Environment](http://www.arcadelearningenvironment.org/) ([https://github.com/mgbellemare/Arcade-Learning-Environment](https://github.com/mgbellemare/Arcade-Learning-Environment))
+* [Arcade Learning Environment](http://www.arcadelearningenvironment.org/) ([https://github.com/sygi/Arcade-Learning-Environment](https://github.com/sygi/Arcade-Learning-Environment) (we use our fork of ALE which draws state of the RAM))
 
 The script `dep_script.sh` can be used to install all dependencies under Ubuntu.
 
 
 # Running
 
-Use the scripts `run_nips.py` or `run_nature.py` to start all the necessary processes:
-
-`$ ./run_nips.py --rom breakout`
-
-`$ ./run_nature.py --rom breakout`
-
-The `run_nips.py` script uses parameters consistent with the original
-NIPS workshop paper.  This code should take 2-4 days to complete.  The
-`run_nature.py` script uses parameters consistent with the Nature
-paper.  The final policies should be better, but it will take 6-10
-days to finish training.
-
-Either script will store output files in a folder prefixed with the
-name of the ROM.  Pickled version of the network objects are stored
-after every epoch.  The file `results.csv` will contain the testing
-output.  You can plot the progress by executing `plot_results.py`:
-
-`$ python plot_results.py breakout_05-28-17-09_0p00025_0p99/results.csv`
-
-After training completes, you can watch the network play using the 
-`ale_run_watch.py` script: 
+TODO: describe all the experiments and how to run the code.
 
-`$ python ale_run_watch.py breakout_05-28-17-09_0p00025_0p99/network_file_99.pkl`
-
-# Performance Tuning
-
-## Theano Configuration
-
-Setting `allow_gc=False` in `THEANO_FLAGS` or in the `.theanorc` file
-significantly improves performance at the expense of a slight increase
-in memory usage on the GPU.
-
-
-# Getting Help
+# See Also
 
-The [deep Q-learning web-forum](https://groups.google.com/forum/#!forum/deep-q-learning)
-can be used for discussion and advice related to deep Q-learning in
-general and this package in particular.
+* https://github.com/spragunr/deep_q_rl
 
-# See Also
+  Original Nathan Sprague implementation of DQN.
 
 * https://sites.google.com/a/deepmind.com/dqn
 

dep_script.sh

@@ -41,7 +41,7 @@ echo "==>installing ALE ..."
 # dependencies ...
 sudo apt-get install libsdl1.2-dev libsdl-gfx1.2-dev libsdl-image1.2-dev cmake
 
-git clone https://github.com/mgbellemare/Arcade-Learning-Environment ALE
+git clone https://github.com/sygi/Arcade-Learning-Environment ALE
 cd ./ALE
 cmake -DUSE_SDL=ON -DUSE_RLGLUE=OFF .
 make -j2