Added logging per Brain of time to update policy, time elapsed during training, time to collect experiences, buffer length, average return per policy #1858
Conversation
|
Looks like you have a couple of test failures. |
eshvk
force-pushed
the
develop-esh-metrics
branch
from
01fb639 to
42099cc
Compare
eshvk
force-pushed
the
develop-esh-metrics
branch
from
66eef6f to
19f244c
Compare
eshvk
force-pushed
the
develop-esh-metrics
branch
2 times, most recently
from
9232e51 to
76b14da
Compare
|
|
||
| self.demonstration_buffer = Buffer() | ||
| self.evaluation_buffer = Buffer() | ||
| self.summary_writer = tf.summary.FileWriter(self.summary_path) |
There was a problem hiding this comment.
Did you test that BC is still training and logging tensorboard ?
There was a problem hiding this comment.
Just tried it, looks like it is still running and logging
| @@ -98,7 +109,10 @@ def get_action(self, curr_info: BrainInfo) -> ActionInfo: | |||
| :param curr_info: Current BrainInfo. | |||
| :return: The ActionInfo given by the policy given the BrainInfo. | |||
| """ | |||
| return self.policy.get_action(curr_info) | |||
| self.trainer_metrics.start_experience_collection_timer() | |||
There was a problem hiding this comment.
In BC trainer that inherits from trainer but no trainer_metrics is used, will this cause problems ?
There was a problem hiding this comment.
No because trainer_metrics is of type TrainerMetrics which is instantiated in the constructor for Trainer, the constructor is called in BCTrainer so the object will exist and the method can be called. We will not write the data because it will be gibberish.
| for brain_name, trainer in self.trainers.items(): | ||
| if brain_name in self.trainer_metrics: | ||
| self.trainer_metrics[brain_name].add_delta_step(delta_time_step) |
There was a problem hiding this comment.
Why not self.trainers[brain_name].trainer_metrics ?
There was a problem hiding this comment.
Can you explain this a bit more? Are you saying why have the if condition?
eshvk
force-pushed
the
develop-esh-metrics
branch
from
76b14da to
67abb7b
Compare
… training, time to collect experiences, buffer length, average return
eshvk
force-pushed
the
develop-esh-metrics
branch
from
67abb7b to
b89bb0f
Compare
|
PTAL @vincentpierre Fixed most of the outstanding issues, squashed, rebased from develop |
Squashing and rebasing against develop