diff --git a/.yamato/com.unity.ml-agents-test.yml b/.yamato/com.unity.ml-agents-test.yml index 012aa1f8e9..c84316e642 100644 --- a/.yamato/com.unity.ml-agents-test.yml +++ b/.yamato/com.unity.ml-agents-test.yml @@ -16,7 +16,8 @@ test_editors: trunk_editor: - version: trunk - enableCodeCoverage: !!bool true + # Workaround for MLA-1596 - need to make sure we load the right results. + enableCodeCoverage: !!bool false testProject: DevProject test_platforms: diff --git a/README.md b/README.md index 880e595c97..84cb810276 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ # Unity ML-Agents Toolkit -[![docs badge](https://img.shields.io/badge/docs-reference-blue.svg)](https://github.com/Unity-Technologies/ml-agents/tree/release_9_docs/docs/) +[![docs badge](https://img.shields.io/badge/docs-reference-blue.svg)](https://github.com/Unity-Technologies/ml-agents/tree/release_10_docs/docs/) [![license badge](https://img.shields.io/badge/license-Apache--2.0-green.svg)](LICENSE) @@ -48,8 +48,8 @@ descriptions of all these features. ## Releases & Documentation -**Our latest, stable release is `Release 9`. Click -[here](https://github.com/Unity-Technologies/ml-agents/tree/release_9_docs/docs/Readme.md) +**Our latest, stable release is `Release 10`. Click +[here](https://github.com/Unity-Technologies/ml-agents/tree/release_10_docs/docs/Readme.md) to get started with the latest release of ML-Agents.** The table below lists all our releases, including our `master` branch which is @@ -67,13 +67,13 @@ under active development and may be unstable. A few helpful guidelines: | **Version** | **Release Date** | **Source** | **Documentation** | **Download** | |:-------:|:------:|:-------------:|:-------:|:------------:| | **master (unstable)** | -- | [source](https://github.com/Unity-Technologies/ml-agents/tree/master) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/master/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/master.zip) | -| **Release 9** | **November 4, 2020** | **[source](https://github.com/Unity-Technologies/ml-agents/tree/release_9)** | **[docs](https://github.com/Unity-Technologies/ml-agents/tree/release_9_docs/docs/Readme.md)** | **[download](https://github.com/Unity-Technologies/ml-agents/archive/release_9.zip)** | +| **Release 10** | **November 18, 2020** | **[source](https://github.com/Unity-Technologies/ml-agents/tree/release_10)** | **[docs](https://github.com/Unity-Technologies/ml-agents/tree/release_10_docs/docs/Readme.md)** | **[download](https://github.com/Unity-Technologies/ml-agents/archive/release_10.zip)** | +| **Release 9** | November 4, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/release_9) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/release_9_docs/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/release_9.zip) | | **Release 8** | October 14, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/release_8) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/release_8_docs/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/release_8.zip) | | **Release 7** | September 16, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/release_7) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/release_7_docs/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/release_7.zip) | | **Release 6** | August 12, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/release_6) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/release_6_docs/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/release_6.zip) | | **Release 5** | July 31, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/release_5) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/release_5_docs/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/release_5.zip) | | **Release 4** | July 15, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/release_4) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/release_4_docs/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/release_4.zip) | -| **Release 3** | June 10, 2020 | [source](https://github.com/Unity-Technologies/ml-agents/tree/release_3) | [docs](https://github.com/Unity-Technologies/ml-agents/tree/release_3_docs/docs/Readme.md) | [download](https://github.com/Unity-Technologies/ml-agents/archive/release_3.zip) | ## Citation diff --git a/com.unity.ml-agents.extensions/Documentation~/Grid-Sensor.md b/com.unity.ml-agents.extensions/Documentation~/Grid-Sensor.md index 1858c835c1..e3dbda315b 100644 --- a/com.unity.ml-agents.extensions/Documentation~/Grid-Sensor.md +++ b/com.unity.ml-agents.extensions/Documentation~/Grid-Sensor.md @@ -36,7 +36,7 @@ These limitations provided the motivation towards the development of the Grid Se An image can be thought of as a matrix of a predefined width (W) and a height (H) and each pixel can be thought of as simply an array of length 3 (in the case of RGB), `[Red, Green, Blue]` holding the different channel information of the color (channel) intensities at that pixel location. Thus an image is just a 3 dimensional matrix of size WxHx3. A Grid Observation can be thought of as a generalization of this setup where in place of a pixel there is a "cell" which is an array of length N representing different channel intensities at that cell position. From a Convolutional Neural Network point of view, the introduction of multiple channels in an "image" isn't a new concept. One such example is using an RGB-Depth image which is used in several robotics applications. The distinction of Grid Observations is what the data within the channels represents. Instead of limiting the channels to color intensities, the channels within a cell of a Grid Observation generalize to any data that can be represented by a single number (float or int). -Before jumping into the details of the Grid Sensor, an important thing to note is the agent performance and qualitatively different behavior over raycasts. Unity MLAgent's comes with a suite of example environments. One in particular, the [Food Collector](https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Learning-Environment-Examples.md#food-collector), has been the focus of the Grid Sensor development. +Before jumping into the details of the Grid Sensor, an important thing to note is the agent performance and qualitatively different behavior over raycasts. Unity MLAgent's comes with a suite of example environments. One in particular, the [Food Collector](https://github.com/Unity-Technologies/ml-agents/tree/release_10_docs/docs/Learning-Environment-Examples.md#food-collector), has been the focus of the Grid Sensor development. The Food Collector environment can be described as: * Set-up: A multi-agent environment where agents compete to collect food. diff --git a/com.unity.ml-agents.extensions/Documentation~/Match3.md b/com.unity.ml-agents.extensions/Documentation~/Match3.md index 74b1ee9e5b..bb314aa137 100644 --- a/com.unity.ml-agents.extensions/Documentation~/Match3.md +++ b/com.unity.ml-agents.extensions/Documentation~/Match3.md @@ -1,9 +1,31 @@ -# Match-3 Game Support +# Match-3 with ML-Agents -We provide some utilities to integrate ML-Agents with Match-3 games. - + -## AbstractBoard class +## Overview +One of the main feedback we get is to illustrate more real game examples using ML-Agents. We are excited to provide an example implementation of Match-3 using ML-Agents and additional utilities to integrate ML-Agents with Match-3 games. + +Our aim is to enable Match-3 teams to leverage ML-Agents to create player agents to learn and play different Match-3 levels. This implementation is intended as a starting point and guide for teams to get started (as there are many nuances with Match-3 for training ML-Agents) and for us to iterate both on the C#, hyperparameters, and trainers to improve ML-Agents for Match-3. + +This implementation includes: + +* C# implementation catered toward a Match-3 setup including concepts around encoding for moves based on [Human Like Playtesting with Deep Learning](https://www.researchgate.net/publication/328307928_Human-Like_Playtesting_with_Deep_Learning) +* An example Match-3 scene with ML-Agents implemented (located under /Project/Assets/ML-Agents/Examples/Match3). More information, on Match-3 example [here](https://github.com/Unity-Technologies/ml-agents/tree/release_10_docs/docs/docs/Learning-Environment-Examples.md#match-3). + +### Feedback +If you are a Match-3 developer and are trying to leverage ML-Agents for this scenario, [we want to hear from you](https://forms.gle/TBsB9jc8WshgzViU9). Additionally, we are also looking for interested Match-3 teams to speak with us for 45 minutes. If you are interested, please indicate that in the [form](https://forms.gle/TBsB9jc8WshgzViU9). If selected, we will provide gift cards as a token of appreciation. + +### Interested in more game templates? +Do you have a type of game you are interested for ML-Agents? If so, please post a [forum issue](https://forum.unity.com/forums/ml-agents.453/) with [GAME TEMPLATE] in the title. + +## Getting started +The C# code for Match-3 exists inside of the extensions package (com.unity.ml-agents.extensions). A good first step would be to familiarize with the extensions package by reading the document [here](com.unity.ml-agents.extensions.md). The second step would be to take a look at how we have implemented the C# code in the example Match-3 scene (located under /Project/Assets/ML-Agents/Examples/match3). Once you have some familiarity, then the next step would be to implement the C# code for Match-3 from the extensions package. + +Additionally, see below for additional technical specifications on the C# code for Match-3. Please note the Match-3 game isn't human playable as implemented and can be only played via training. + +## Technical specifications for Match-3 with ML-Agents + +### AbstractBoard class The `AbstractBoard` is the bridge between ML-Agents and your game. It allows ML-Agents to * ask your game what the "color" of a cell is * ask whether the cell is a "special" piece type or not @@ -14,27 +36,27 @@ These are handled by implementing the `GetCellType()`, `IsMoveValid()`, and `Mak The AbstractBoard also tracks the number of rows, columns, and potential piece types that the board can have. -#### `public abstract int GetCellType(int row, int col)` +##### `public abstract int GetCellType(int row, int col)` Returns the "color" of piece at the given row and column. This should be between 0 and NumCellTypes-1 (inclusive). The actual order of the values doesn't matter. -#### `public abstract int GetSpecialType(int row, int col)` +##### `public abstract int GetSpecialType(int row, int col)` Returns the special type of the piece at the given row and column. This should be between 0 and NumSpecialTypes (inclusive). The actual order of the values doesn't matter. -#### `public abstract bool IsMoveValid(Move m)` +##### `public abstract bool IsMoveValid(Move m)` Check whether the particular `Move` is valid for the game. The actual results will depend on the rules of the game, but we provide the `SimpleIsMoveValid()` method that handles basic match3 rules with no special or immovable pieces. -#### `public abstract bool MakeMove(Move m)` +##### `public abstract bool MakeMove(Move m)` Instruct the game to make the given move. Returns true if the move was made. Note that during training, a move that was marked as invalid may occasionally still be requested. If this happens, it is safe to do nothing and request another move. -## Move struct +### Move struct The Move struct encapsulates a swap of two adjacent cells. You can get the number of potential moves for a board of a given size with. `Move.NumPotentialMoves(NumRows, NumColumns)`. There are two helper functions to create a new `Move`: @@ -43,7 +65,7 @@ iterate over all potential moves for the board by looping from 0 to `Move.NumPot * `public static Move FromPositionAndDirection(int row, int col, Direction dir, int maxRows, int maxCols)` creates a `Move` from a row, column, and direction (and board size). -## `Match3Sensor` and `Match3SensorComponent` classes +#### `Match3Sensor` and `Match3SensorComponent` classes The `Match3Sensor` generates observations about the state using the `AbstractBoard` interface. You can choose whether to use vector or "visual" observations; in theory, visual observations should perform better because they are 2-dimensional like the board, but we need to experiment more on this. @@ -51,14 +73,14 @@ better because they are 2-dimensional like the board, but we need to experiment A `Match3SensorComponent` generates a `Match3Sensor` at runtime, and should be added to the same GameObject as your `Agent` implementation. You do not need to write any additional code to use them. -## `Match3Actuator` and `Match3ActuatorComponent` classes +#### `Match3Actuator` and `Match3ActuatorComponent` classes The `Match3Actuator` converts actions from training or inference into a `Move` that is sent to` AbstractBoard.MakeMove()` It also checks `AbstractBoard.IsMoveValid` for each potential move and uses this to set the action mask for Agent. A `Match3ActuatorComponent` generates a `Match3Actuator` at runtime, and should be added to the same GameObject as your `Agent` implementation. You do not need to write any additional code to use them. -# Setting up match-3 simulation +### Setting up Match-3 simulation * Implement the `AbstractBoard` methods to integrate with your game. * Give the `Agent` rewards when it does what you want it to (match multiple pieces in a row, clears pieces of a certain type, etc). diff --git a/com.unity.ml-agents.extensions/Documentation~/com.unity.ml-agents.extensions.md b/com.unity.ml-agents.extensions/Documentation~/com.unity.ml-agents.extensions.md index 4214418d74..b3038c314b 100644 --- a/com.unity.ml-agents.extensions/Documentation~/com.unity.ml-agents.extensions.md +++ b/com.unity.ml-agents.extensions/Documentation~/com.unity.ml-agents.extensions.md @@ -28,14 +28,14 @@ The ML-Agents Extensions package is not currently available in the Package Manag recommended ways to install the package: ### Local Installation -[Clone the repository](../../docs/Installation.md#clone-the-ml-agents-toolkit-repository-optional) and follow the -[Local Installation for Development](../../docs/Installation.md#advanced-local-installation-for-development-1) +[Clone the repository](https://github.com/Unity-Technologies/ml-agents/tree/release_10_docs/docs/Installation.md#clone-the-ml-agents-toolkit-repository-optional) and follow the +[Local Installation for Development](https://github.com/Unity-Technologies/ml-agents/tree/release_10_docs/docs/Installation.md#advanced-local-installation-for-development-1) directions (substituting `com.unity.ml-agents.extensions` for the package name). ### Github via Package Manager In Unity 2019.4 or later, open the Package Manager, hit the "+" button, and select "Add package from git URL". -![Package Manager git URL](../../docs/images/unity_package_manager_git_url.png) +![Package Manager git URL](https://github.com/Unity-Technologies/ml-agents/tree/release_10_docs/docs/images/unity_package_manager_git_url.png) In the dialog that appears, enter ``` git+https://github.com/Unity-Technologies/ml-agents.git?path=com.unity.ml-agents.extensions @@ -60,4 +60,4 @@ following versions of the Unity Editor: none ## Need Help? -The main [README](../../README.md) contains links for contacting the team or getting support. +The main [README](https://github.com/Unity-Technologies/ml-agents/tree/release_10_docs/README.md) contains links for contacting the team or getting support. diff --git a/com.unity.ml-agents/CHANGELOG.md b/com.unity.ml-agents/CHANGELOG.md index e5db334301..8c7c782714 100755 --- a/com.unity.ml-agents/CHANGELOG.md +++ b/com.unity.ml-agents/CHANGELOG.md @@ -19,7 +19,6 @@ and this project adheres to ### Bug Fixes #### com.unity.ml-agents (C#) #### ml-agents / ml-agents-envs / gym-unity (Python) - - Remove extra period after "Training" in console log. (#4674) ## [1.6.0-preview] - 2020-11-18 @@ -50,6 +49,7 @@ Previously, this would result in an infinite loop and cause the editor to hang. - Fixed an issue where runs could not be resumed when using TensorFlow and Ghost Training. (#4593) - Change the tensor type of step count from int32 to int64 to address the overflow issue when step goes larger than 2^31. Previous Tensorflow checkpoints will become incompatible and cannot be loaded. (#4607) +- Remove extra period after "Training" in console log. (#4674) ## [1.5.0-preview] - 2020-10-14 diff --git a/com.unity.ml-agents/Documentation~/com.unity.ml-agents.md b/com.unity.ml-agents/Documentation~/com.unity.ml-agents.md index edde72926b..e5c368d284 100755 --- a/com.unity.ml-agents/Documentation~/com.unity.ml-agents.md +++ b/com.unity.ml-agents/Documentation~/com.unity.ml-agents.md @@ -114,7 +114,7 @@ a number of ways to [connect with us] including our [ML-Agents Forum]. [unity ML-Agents Toolkit]: https://github.com/Unity-Technologies/ml-agents [unity inference engine]: https://docs.unity3d.com/Packages/com.unity.barracuda@latest/index.html [package manager documentation]: https://docs.unity3d.com/Manual/upm-ui-install.html -[installation instructions]: https://github.com/Unity-Technologies/ml-agents/blob/release_9_docs/docs/Installation.md +[installation instructions]: https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/docs/Installation.md [github repository]: https://github.com/Unity-Technologies/ml-agents [python package]: https://github.com/Unity-Technologies/ml-agents [execution order of event functions]: https://docs.unity3d.com/Manual/ExecutionOrder.html diff --git a/com.unity.ml-agents/Runtime/Academy.cs b/com.unity.ml-agents/Runtime/Academy.cs index 5d0bbb3bc2..593feb646f 100644 --- a/com.unity.ml-agents/Runtime/Academy.cs +++ b/com.unity.ml-agents/Runtime/Academy.cs @@ -20,7 +20,7 @@ * API. For more information on each of these entities, in addition to how to * set-up a learning environment and train the behavior of characters in a * Unity scene, please browse our documentation pages on GitHub: - * https://github.com/Unity-Technologies/ml-agents/tree/release_9_docs/docs/ + * https://github.com/Unity-Technologies/ml-agents/tree/release_10_docs/docs/ */ namespace Unity.MLAgents @@ -61,7 +61,7 @@ void FixedUpdate() /// fall back to inference or heuristic decisions. (You can also set agents to always use /// inference or heuristics.) /// - [HelpURL("https://github.com/Unity-Technologies/ml-agents/tree/release_9_docs/" + + [HelpURL("https://github.com/Unity-Technologies/ml-agents/tree/release_10_docs/" + "docs/Learning-Environment-Design.md")] public class Academy : IDisposable { diff --git a/com.unity.ml-agents/Runtime/Actuators/IActionReceiver.cs b/com.unity.ml-agents/Runtime/Actuators/IActionReceiver.cs index 9cfd96ed25..c5978a2552 100644 --- a/com.unity.ml-agents/Runtime/Actuators/IActionReceiver.cs +++ b/com.unity.ml-agents/Runtime/Actuators/IActionReceiver.cs @@ -160,7 +160,7 @@ public interface IActionReceiver /// /// See [Agents - Actions] for more information on masking actions. /// - /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_9_docs/docs/Learning-Environment-Design-Agents.md#actions + /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/docs/Learning-Environment-Design-Agents.md#actions /// /// void WriteDiscreteActionMask(IDiscreteActionMask actionMask); diff --git a/com.unity.ml-agents/Runtime/Actuators/IDiscreteActionMask.cs b/com.unity.ml-agents/Runtime/Actuators/IDiscreteActionMask.cs index 4d37977a3e..f195fa3b58 100644 --- a/com.unity.ml-agents/Runtime/Actuators/IDiscreteActionMask.cs +++ b/com.unity.ml-agents/Runtime/Actuators/IDiscreteActionMask.cs @@ -17,7 +17,7 @@ public interface IDiscreteActionMask /// /// See [Agents - Actions] for more information on masking actions. /// - /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_9_docs/docs/Learning-Environment-Design-Agents.md#actions + /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/docs/Learning-Environment-Design-Agents.md#actions /// /// The branch for which the actions will be masked. /// The indices of the masked actions. diff --git a/com.unity.ml-agents/Runtime/Agent.cs b/com.unity.ml-agents/Runtime/Agent.cs index a5a018d0d2..a6bd21b7b3 100644 --- a/com.unity.ml-agents/Runtime/Agent.cs +++ b/com.unity.ml-agents/Runtime/Agent.cs @@ -148,13 +148,13 @@ public void CopyActions(ActionBuffers actionBuffers) /// [OnDisable()]: https://docs.unity3d.com/ScriptReference/MonoBehaviour.OnDisable.html] /// [OnBeforeSerialize()]: https://docs.unity3d.com/ScriptReference/MonoBehaviour.OnBeforeSerialize.html /// [OnAfterSerialize()]: https://docs.unity3d.com/ScriptReference/MonoBehaviour.OnAfterSerialize.html - /// [Agents]: https://github.com/Unity-Technologies/ml-agents/blob/release_9_docs/docs/Learning-Environment-Design-Agents.md - /// [Reinforcement Learning in Unity]: https://github.com/Unity-Technologies/ml-agents/blob/release_9_docs/docs/Learning-Environment-Design.md + /// [Agents]: https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/docs/Learning-Environment-Design-Agents.md + /// [Reinforcement Learning in Unity]: https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/docs/Learning-Environment-Design.md /// [Unity ML-Agents Toolkit]: https://github.com/Unity-Technologies/ml-agents - /// [Unity ML-Agents Toolkit manual]: https://github.com/Unity-Technologies/ml-agents/blob/release_9_docs/docs/Readme.md + /// [Unity ML-Agents Toolkit manual]: https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/docs/Readme.md /// /// - [HelpURL("https://github.com/Unity-Technologies/ml-agents/blob/release_9_docs/" + + [HelpURL("https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/" + "docs/Learning-Environment-Design-Agents.md")] [Serializable] [RequireComponent(typeof(BehaviorParameters))] @@ -639,8 +639,8 @@ public int CompletedEpisodes /// for information about mixing reward signals from curiosity and Generative Adversarial /// Imitation Learning (GAIL) with rewards supplied through this method. /// - /// [Agents - Rewards]: https://github.com/Unity-Technologies/ml-agents/blob/release_9_docs/docs/Learning-Environment-Design-Agents.md#rewards - /// [Reward Signals]: https://github.com/Unity-Technologies/ml-agents/blob/release_9_docs/docs/ML-Agents-Overview.md#a-quick-note-on-reward-signals + /// [Agents - Rewards]: https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/docs/Learning-Environment-Design-Agents.md#rewards + /// [Reward Signals]: https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/docs/ML-Agents-Overview.md#a-quick-note-on-reward-signals /// /// The new value of the reward. public void SetReward(float reward) @@ -669,8 +669,8 @@ public void SetReward(float reward) /// for information about mixing reward signals from curiosity and Generative Adversarial /// Imitation Learning (GAIL) with rewards supplied through this method. /// - /// [Agents - Rewards]: https://github.com/Unity-Technologies/ml-agents/blob/release_9_docs/docs/Learning-Environment-Design-Agents.md#rewards - /// [Reward Signals]: https://github.com/Unity-Technologies/ml-agents/blob/release_9_docs/docs/ML-Agents-Overview.md#a-quick-note-on-reward-signals + /// [Agents - Rewards]: https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/docs/Learning-Environment-Design-Agents.md#rewards + /// [Reward Signals]: https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/docs/ML-Agents-Overview.md#a-quick-note-on-reward-signals /// /// Incremental reward value. public void AddReward(float increment) @@ -848,8 +848,8 @@ public virtual void Initialize() { } /// implementing a simple heuristic function can aid in debugging agent actions and interactions /// with its environment. /// - /// [Demonstration Recorder]: https://github.com/Unity-Technologies/ml-agents/blob/release_9_docs/docs/Learning-Environment-Design-Agents.md#recording-demonstrations - /// [Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_9_docs/docs/Learning-Environment-Design-Agents.md#actions + /// [Demonstration Recorder]: https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/docs/Learning-Environment-Design-Agents.md#recording-demonstrations + /// [Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/docs/Learning-Environment-Design-Agents.md#actions /// [GameObject]: https://docs.unity3d.com/Manual/GameObjects.html /// /// @@ -1099,7 +1099,7 @@ void ResetSensors() /// For more information about observations, see [Observations and Sensors]. /// /// [GameObject]: https://docs.unity3d.com/Manual/GameObjects.html - /// [Observations and Sensors]: https://github.com/Unity-Technologies/ml-agents/blob/release_9_docs/docs/Learning-Environment-Design-Agents.md#observations-and-sensors + /// [Observations and Sensors]: https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/docs/Learning-Environment-Design-Agents.md#observations-and-sensors /// public virtual void CollectObservations(VectorSensor sensor) { @@ -1130,7 +1130,7 @@ public ReadOnlyCollection GetObservations() /// /// See [Agents - Actions] for more information on masking actions. /// - /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_9_docs/docs/Learning-Environment-Design-Agents.md#actions + /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/docs/Learning-Environment-Design-Agents.md#actions /// /// public virtual void WriteDiscreteActionMask(IDiscreteActionMask actionMask) @@ -1205,7 +1205,7 @@ public virtual void WriteDiscreteActionMask(IDiscreteActionMask actionMask) /// /// For more information about implementing agent actions see [Agents - Actions]. /// - /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_9_docs/docs/Learning-Environment-Design-Agents.md#actions + /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/docs/Learning-Environment-Design-Agents.md#actions /// /// /// Struct containing the buffers of actions to be executed at this step. diff --git a/com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs b/com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs index c471f71418..74bc034ba5 100644 --- a/com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs +++ b/com.unity.ml-agents/Runtime/Demonstrations/DemonstrationRecorder.cs @@ -19,7 +19,7 @@ namespace Unity.MLAgents.Demonstrations /// See [Imitation Learning - Recording Demonstrations] for more information. /// /// [GameObject]: https://docs.unity3d.com/Manual/GameObjects.html - /// [Imitation Learning - Recording Demonstrations]: https://github.com/Unity-Technologies/ml-agents/blob/release_9_docs/docs//Learning-Environment-Design-Agents.md#recording-demonstrations + /// [Imitation Learning - Recording Demonstrations]: https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/docs//Learning-Environment-Design-Agents.md#recording-demonstrations /// [RequireComponent(typeof(Agent))] [AddComponentMenu("ML Agents/Demonstration Recorder", (int)MenuGroup.Default)] diff --git a/com.unity.ml-agents/Runtime/DiscreteActionMasker.cs b/com.unity.ml-agents/Runtime/DiscreteActionMasker.cs index 3a7dcb96c6..20bf208c3d 100644 --- a/com.unity.ml-agents/Runtime/DiscreteActionMasker.cs +++ b/com.unity.ml-agents/Runtime/DiscreteActionMasker.cs @@ -33,7 +33,7 @@ internal DiscreteActionMasker(IDiscreteActionMask actionMask) /// /// See [Agents - Actions] for more information on masking actions. /// - /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_9_docs/docs/Learning-Environment-Design-Agents.md#actions + /// [Agents - Actions]: https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/docs/Learning-Environment-Design-Agents.md#actions /// /// The branch for which the actions will be masked. /// The indices of the masked actions. diff --git a/com.unity.ml-agents/Runtime/Sensors/ObservationWriter.cs b/com.unity.ml-agents/Runtime/Sensors/ObservationWriter.cs index 5b0b96b3df..2e1f0f2336 100644 --- a/com.unity.ml-agents/Runtime/Sensors/ObservationWriter.cs +++ b/com.unity.ml-agents/Runtime/Sensors/ObservationWriter.cs @@ -209,9 +209,11 @@ public void Add(Quaternion quat, int writeOffset = 0) } } + /// + /// Provides extension methods for the ObservationWriter. + /// public static class ObservationWriterExtension { - /// /// Writes a Texture2D into a ObservationWriter. /// diff --git a/docs/Installation-Anaconda-Windows.md b/docs/Installation-Anaconda-Windows.md index 7b27e63042..39e67a9d42 100644 --- a/docs/Installation-Anaconda-Windows.md +++ b/docs/Installation-Anaconda-Windows.md @@ -123,10 +123,10 @@ commands in an Anaconda Prompt _(if you open a new prompt, be sure to activate the ml-agents Conda environment by typing `activate ml-agents`)_: ```sh -git clone --branch release_9 https://github.com/Unity-Technologies/ml-agents.git +git clone --branch release_10 https://github.com/Unity-Technologies/ml-agents.git ``` -The `--branch release_9` option will switch to the tag of the latest stable +The `--branch release_10` option will switch to the tag of the latest stable release. Omitting that will get the `master` branch which is potentially unstable. diff --git a/docs/Installation.md b/docs/Installation.md index 324ded2e4c..9e03c70935 100644 --- a/docs/Installation.md +++ b/docs/Installation.md @@ -53,10 +53,10 @@ example environments and training configurations to experiment with them (some of our tutorials / guides assume you have access to our example environments). ```sh -git clone --branch release_9 https://github.com/Unity-Technologies/ml-agents.git +git clone --branch release_10 https://github.com/Unity-Technologies/ml-agents.git ``` -The `--branch release_9` option will switch to the tag of the latest stable +The `--branch release_10` option will switch to the tag of the latest stable release. Omitting that will get the `master` branch which is potentially unstable. @@ -64,7 +64,7 @@ unstable. You will need to clone the repository if you plan to modify or extend the ML-Agents Toolkit for your purposes. If you plan to contribute those changes -back, make sure to clone the `master` branch (by omitting `--branch release_9` +back, make sure to clone the `master` branch (by omitting `--branch release_10` from the command above). See our [Contributions Guidelines](../com.unity.ml-agents/CONTRIBUTING.md) for more information on contributing to the ML-Agents Toolkit. diff --git a/docs/Learning-Environment-Examples.md b/docs/Learning-Environment-Examples.md index 178ab5ecaf..141a98c1e1 100644 --- a/docs/Learning-Environment-Examples.md +++ b/docs/Learning-Environment-Examples.md @@ -538,7 +538,6 @@ you would like to contribute environments, please see our - Benchmark Mean Reward: 1.75 ## Match 3 - ![Match 3](images/match3.png) - Set-up: Simple match-3 game. Matched pieces are removed, and remaining pieces diff --git a/docs/Training-ML-Agents.md b/docs/Training-ML-Agents.md index 67fb4d18f7..ae3ddc42c4 100644 --- a/docs/Training-ML-Agents.md +++ b/docs/Training-ML-Agents.md @@ -58,7 +58,7 @@ mlagents-learn --env= --run-id= where -- `` is the file path of the trainer configuration yaml. +- `` is the file path of the trainer configuration YAML. This contains all the hyperparameter values. We offer a detailed guide on the structure of this file and the meaning of the hyperparameters (and advice on how to set them) in the dedicated @@ -138,14 +138,6 @@ flags for `mlagents-learn` that control the training configurations: - ``: defines the training hyperparameters for each Behavior in the scene, and the set-ups for the environment parameters (Curriculum Learning and Environment Parameter Randomization) -- `--num-envs`: number of concurrent Unity instances to use during training - -Reminder that a detailed description of all command-line options can be found by -using the help utility: - -```sh -mlagents-learn --help -``` It is important to highlight that successfully training a Behavior in the ML-Agents Toolkit involves tuning the training hyperparameters and @@ -172,7 +164,6 @@ add typically has its own training configurations. For instance: demonstrations.) - Use self-play? (Assuming your environment includes multiple agents.) - The trainer config file, ``, determines the features you will use during training, and the answers to the above questions will dictate its contents. The rest of this guide breaks down the different sub-sections of the trainer config file @@ -185,6 +176,57 @@ an old set of configuration files (trainer config, curriculum, and sampler files format, a script has been provided. Run `python -m mlagents.trainers.upgrade_config -h` in your console to see the script's usage. +### Adding CLI Arguments to the Training Configuration file + +Additionally, within the training configuration YAML file, you can also add the +CLI arguments (such as `--num-envs`). + +Reminder that a detailed description of all the CLI arguments can be found by +using the help utility: + +```sh +mlagents-learn --help +``` + +These additional CLI arguments are grouped into environment, engine and checkpoint. The available settings and example values are shown below. + +#### Environment settings + +```yaml +env_settings: + env_path: FoodCollector + env_args: null + base_port: 5005 + num_envs: 1 + seed: -1 +``` + +#### Engine settings + +```yaml +engine_settings: + width: 84 + height: 84 + quality_level: 5 + time_scale: 20 + target_frame_rate: -1 + capture_frame_rate: 60 + no_graphics: false +``` + +#### Checkpoint settings + +```yaml +checkpoint_settings: + run_id: foodtorch + initialize_from: null + load_model: false + resume: false + force: true + train_model: false + inference: false +``` + ### Behavior Configurations The primary section of the trainer config file is a diff --git a/docs/Training-on-Amazon-Web-Service.md b/docs/Training-on-Amazon-Web-Service.md index 2a3f97a3df..84a30e5166 100644 --- a/docs/Training-on-Amazon-Web-Service.md +++ b/docs/Training-on-Amazon-Web-Service.md @@ -69,7 +69,7 @@ After launching your EC2 instance using the ami and ssh into it: 2. Clone the ML-Agents repo and install the required Python packages ```sh - git clone --branch release_9 https://github.com/Unity-Technologies/ml-agents.git + git clone --branch release_10 https://github.com/Unity-Technologies/ml-agents.git cd ml-agents/ml-agents/ pip3 install -e . ``` diff --git a/docs/Unity-Inference-Engine.md b/docs/Unity-Inference-Engine.md index 4e99274c2b..d7b46f8f04 100644 --- a/docs/Unity-Inference-Engine.md +++ b/docs/Unity-Inference-Engine.md @@ -48,9 +48,9 @@ The ML-Agents Toolkit only supports the models created with our trainers. Model loading expects certain conventions for constants and tensor names. While it is possible to construct a model that follows these conventions, we don't provide any additional help for this. More details can be found in -[TensorNames.cs](https://github.com/Unity-Technologies/ml-agents/blob/release_9_docs/com.unity.ml-agents/Runtime/Inference/TensorNames.cs) +[TensorNames.cs](https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/com.unity.ml-agents/Runtime/Inference/TensorNames.cs) and -[BarracudaModelParamLoader.cs](https://github.com/Unity-Technologies/ml-agents/blob/release_9_docs/com.unity.ml-agents/Runtime/Inference/BarracudaModelParamLoader.cs). +[BarracudaModelParamLoader.cs](https://github.com/Unity-Technologies/ml-agents/blob/release_10_docs/com.unity.ml-agents/Runtime/Inference/BarracudaModelParamLoader.cs). If you wish to run inference on an externally trained model, you should use Barracuda directly, instead of trying to run it through ML-Agents. diff --git a/docs/images/match3.png b/docs/images/match3.png index ea3e1b40c5..0091d798b7 100644 Binary files a/docs/images/match3.png and b/docs/images/match3.png differ diff --git a/utils/make_readme_table.py b/utils/make_readme_table.py index d37c6fe77c..58dcea1a08 100644 --- a/utils/make_readme_table.py +++ b/utils/make_readme_table.py @@ -75,6 +75,7 @@ def display_name(self) -> str: ReleaseInfo("release_7", "1.4.0", "0.20.0", "September 16, 2020"), ReleaseInfo("release_8", "1.5.0", "0.21.0", "October 14, 2020"), ReleaseInfo("release_9", "1.5.0", "0.21.1", "November 4, 2020"), + ReleaseInfo("release_10", "1.6.0", "0.22.0", "November 18, 2020"), ] MAX_DAYS = 150 # do not print releases older than this many days