You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In ML.NET LightGbm wrapper, the default EvaluationMetric is set to EvaluateMetricType.Error for multiclass, EvaluationMetricType.LogLoss for binary, and so on. On the other hand, in standalone LightGbm, the default evaluation metric is "", which means that LightGbm will automatically select the default metric for the given objective function.
This leads to inconsistent behavior from the user's perspective: If a user specified EvaluationMetric = EvaluateMetricType.Default, the parameter passed to LightGbm would be the empty string "" but if they do not specify EvaluationMetric at all, the parameter passed to LightGbm would be Error for multiclass, LogLoss for binary, and so on.
We need to investigate whether these metrics are indeed the defaults for the respective objective functions in LightGbm, and if they are not, then change the defaults in ML.NET to conform to standalone LightGbm. Note that this would be a breaking change.
The text was updated successfully, but these errors were encountered:
Found while investigating fix for #3761
In ML.NET LightGbm wrapper, the default EvaluationMetric is set to EvaluateMetricType.Error for multiclass, EvaluationMetricType.LogLoss for binary, and so on. On the other hand, in standalone LightGbm, the default evaluation metric is "", which means that LightGbm will automatically select the default metric for the given objective function.
This leads to inconsistent behavior from the user's perspective: If a user specified EvaluationMetric = EvaluateMetricType.Default, the parameter passed to LightGbm would be the empty string "" but if they do not specify EvaluationMetric at all, the parameter passed to LightGbm would be Error for multiclass, LogLoss for binary, and so on.
We need to investigate whether these metrics are indeed the defaults for the respective objective functions in LightGbm, and if they are not, then change the defaults in ML.NET to conform to standalone LightGbm. Note that this would be a breaking change.
The text was updated successfully, but these errors were encountered: