Skip to content

Fix environment factory pickling on Windows #1912

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 9, 2019

Conversation

harperj
Copy link
Contributor

@harperj harperj commented Apr 8, 2019

SubprocessUnityEnvironment sends an environment factory function to
each worker which it can use to create a UnityEnvironment to interact
with. We use Python's standard multiprocessing library, which pickles
all data sent to the subprocess. The built-in pickle library doesn't
pickle function objects on Windows machines (tested with Python 3.6 on
Windows 10 Pro).

This PR adds cloudpickle as a dependency in order to serialize the
environment factory. Other implementations of subprocess environments
do the same:
https://github.com/openai/baselines/blob/master/baselines/common/vec_env/subproc_vec_env.py

SubprocessUnityEnvironment sends an environment factory function to
each worker which it can use to create a UnityEnvironment to interact
with. We use Python's standard multiprocessing library, which pickles
all data sent to the subprocess.  The built-in pickle library doesn't
pickle function objects on Windows machines (tested with Python 3.6 on
Windows 10 Pro).

This PR adds cloudpickle as a dependency in order to serialize the
environment factory. Other implementations of subprocess environments
do the same:
https://github.com/openai/baselines/blob/master/baselines/common/vec_env/subproc_vec_env.py
@harperj harperj requested review from eshvk and xiaomaogy April 8, 2019 18:21
Copy link
Contributor

@xiaomaogy xiaomaogy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xiaomaogy
Copy link
Contributor

@harperj Just to make sure, does the model get saved under Windows for these two cases?

  1. ctrl + c interrupt.
  2. Finish the training normally.

Also have you tested the Linux environment for these two cases?

@SetoKaiba
Copy link

SetoKaiba commented Apr 9, 2019

  1. Stop from Unity Editor.
    Now: Model is not saved.
    Previous: Model is saved. It's saved even just after the training begins for seconds.

  2. Run out the whole training for max_steps.
    Now: The Unity Editor is frozen. And the training is never finished. The checkpoints are saved. But the nn model is not. Press Ctrl C and the Unity Editor is not frozen any more. The nn model is saved.
    Previous: The Unity Editor will stop playing. The training is finished. Both checkpoints and nn model are saved. You don't have to press Ctrl C.

  3. Stop with Ctrl C.
    Now: Model is not saved.
    Previous: Model is saved. It's saved even just after the training begins for seconds.

Test Environment: Windows 10

@xiaomaogy
Copy link
Contributor

@SetoKaiba Thanks for testing this for us! @harperj Just showed me a fix yesterday afternoon, which works for case 2 and case 3 now, I believe he will test the case 1 and then update this branch.

@xiaomaogy xiaomaogy self-requested a review April 9, 2019 16:45
Copy link
Contributor

@xiaomaogy xiaomaogy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes needed to fix the ctrl+c and max_step reached errors.

@harperj
Copy link
Contributor Author

harperj commented Apr 9, 2019

@xiaomaogy @SetoKaiba Those are different (unrelated) issues which I'll shortly make subsequent PRs for.

@harperj harperj merged commit 8ee84fe into release-v0.8 Apr 9, 2019
@ervteng ervteng deleted the release-v08-fixpickle branch July 9, 2019 22:01
LeSphax pushed a commit to LeSphax/ml-agents-1 that referenced this pull request May 3, 2020
SubprocessUnityEnvironment sends an environment factory function to
each worker which it can use to create a UnityEnvironment to interact
with. We use Python's standard multiprocessing library, which pickles
all data sent to the subprocess.  The built-in pickle library doesn't
pickle function objects on Windows machines (tested with Python 3.6 on
Windows 10 Pro).

This PR adds cloudpickle as a dependency in order to serialize the
environment factory. Other implementations of subprocess environments
do the same:
https://github.com/openai/baselines/blob/master/baselines/common/vec_env/subproc_vec_env.py
@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 18, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants