Speed-up to O(1) from O(N) of the computation of each return in REINFORCE

[Chris1nexus]

Fix

The function finish_episode() in the reinforce.py script computes returns by insertion at the beginning of a list.
This is an expensive computation in python as shown by the official docs, since it requires O(N) time to shift all its successor elements.
The same can be achieved but with O(1) time complexity with the python deque data structure, natively supported by python.
To verify the above claims, i tested insertion of 100k elements at the beginning of a list and a deque.
The results are:

• list insertion at the beginning -> 1 second
• deque insertion at the beginning -> 0.004 seconds

Result

Hence, this is a 250 x speed-up, which becomes relevant when the number of episodes becomes large, as this computation is done at the end of every episode.

Python docs Reference: https://docs.python.org/3/tutorial/datastructures.html#using-lists-as-queues

Test

As per the contribution guidelines, the following tests have been run and successfully completed.
./run_python_examples.sh "install_deps,reinforcement_learning,clean"

[netlify]

✅ Deploy Preview for pytorch-examples-preview canceled.

Name	Link
🔨 Latest commit	f29a6a8
🔍 Latest deploy log	https://app.netlify.com/sites/pytorch-examples-preview/deploys/634cc7c7727ad00008c25ab5

[hudeven]

LGTM. The improvement is impressive!

[Chris1nexus]

Thanks for the quick review!