You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
System Information (please complete the following information):
OS & Version: Windows 11 [Version 10.0.22621.675]
ML.NET Version: ML.NT v2.0.0-preview.22525.3
.NET Version: .NET 6.0.9
Describe the bug
I created a custom class that implements the Microsoft.ML.AutoML.IMonitor to monitor my training. It's ReportRunningTrial method has the following simple implementation: _logger.LogInformation($"{"Running".PadLeft(9)} Trial #{setting.TrialId.ToString().PadLeft(4)} - Pipeline: {_pipeline.ToString(setting.Parameter).PadRight(70)}");
The other, ReportBestTrial, ReportCompletedTrial and ReportFailTrial methods look very similar.
When I run my training it looks like that the ReportRunningTrial method's pipeline object wasn't updated, so it doesn't show the pipeline of the current trial.
Make sure that you allow more than one type of (regression) algorithms
Run the experiment
Check the output: you will see that the settings.TrialId property values look good, but the _pipeline.ToString(setting.Parameter) returns an invalid value in the ReportRunningTrial method.
Expected behavior
The _pipeline.ToString(setting.Parameter) should return the just-started trial's pipeline.
Screenshots, Code, Sample Projects
Additional context
You can see the problem on the screenshot:
Running Trial 0's pipeline looks like it was FastTreeRegression, but in reality it's FastForestRegression.
Running Trial 1's pipeline looks like it was FastForestRegression, but it's FastTreeRegression.
Running Trial 2's pipeline looks like it was FastForestRegression, but it's LightGbmRegression.
etc.
Edit: After testing a little more I'm not sure anymore which methods of the IMonitor interface report the pipeline wrong, it is possible that ReportRunningTrial is the correct one and the others are incorrect.
The text was updated successfully, but these errors were encountered:
After I looked into the ML.NET code a little more, my current understanding is that we don't know the last estimator when the IMonitor's ReportRunningTrial() event is called. The last estimator is set only after we call runner.RunAsync() and that's why the "Running Trial" lines in my log show the pipeline incorrectly (they have a previously set value).
I modified my event handler code to not show the pipeline at that point, but I think it would be a little better if the TrialSettings class that is passed to all IMonitor event handlers would contain a nullable property with the last (or all) estimator(s).
System Information (please complete the following information):
Describe the bug
I created a custom class that implements the
Microsoft.ML.AutoML.IMonitor
to monitor my training. It'sReportRunningTrial
method has the following simple implementation:_logger.LogInformation($"{"Running".PadLeft(9)} Trial #{setting.TrialId.ToString().PadLeft(4)} - Pipeline: {_pipeline.ToString(setting.Parameter).PadRight(70)}");
The other,
ReportBestTrial
,ReportCompletedTrial
andReportFailTrial
methods look very similar.When I run my training it looks like that the
ReportRunningTrial
method's pipeline object wasn't updated, so it doesn't show the pipeline of the current trial.To Reproduce
Steps to reproduce the behavior:
settings.TrialId
property values look good, but the_pipeline.ToString(setting.Parameter)
returns an invalid value in theReportRunningTrial
method.Expected behavior
The
_pipeline.ToString(setting.Parameter)
should return the just-started trial's pipeline.Screenshots, Code, Sample Projects

Additional context
You can see the problem on the screenshot:
Running Trial 0's pipeline looks like it was
FastTreeRegression
, but in reality it'sFastForestRegression
.Running Trial 1's pipeline looks like it was
FastForestRegression
, but it'sFastTreeRegression
.Running Trial 2's pipeline looks like it was
FastForestRegression
, but it'sLightGbmRegression
.etc.
Edit: After testing a little more I'm not sure anymore which methods of the
IMonitor
interface report the pipeline wrong, it is possible thatReportRunningTrial
is the correct one and the others are incorrect.The text was updated successfully, but these errors were encountered: