Skip to content

Prediction engine options #5964

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Oct 12, 2021
Merged

Conversation

michaelgsharp
Copy link
Member

Fixes #5945

Currently when you dispose a prediction engine it also disposes the underlying model. This means that you are unable to cache a model to use multiple times if you want to create/dispose the prediction engine. This change makes it so the Prediction Engine no longer automatically disposes the model, allowing you to cache the model without worry about it being automatically disposed. You do now manually have to dispose of it.

@michaelgsharp michaelgsharp requested a review from a team October 8, 2021 18:47
@michaelgsharp
Copy link
Member Author

@eerhardt I think this is a good change, but I would like your opinion as well. Besides the linked issue, there are also some email threads if you would like more information.

@sharwell
Copy link
Member

sharwell commented Oct 8, 2021

💡 The non-breaking way to approach this is to add a constructor (or similar) with a bool ownsTransformer parameter similar to XmlDictionaryWriter, or a bool leaveOpen parameter similar to BinaryReader.

There are examples of both directions of this pattern (true means keep open on dispose, or true means close on dispose). @stephentoub or @terrajobst are best positioned to state the current recommended semantics.

@michaelgsharp
Copy link
Member Author

Thats good to know. @stephentoub or @terrajobst do you have a preference?

@michaelgsharp
Copy link
Member Author

Looks like in our code base we have a couple of places where we are already doing it the same way as the XML. I'll convert it to that way, but if we want it the other way its fast to change.

@codecov
Copy link

codecov bot commented Oct 8, 2021

Codecov Report

Merging #5964 (2189839) into main (54a3cb9) will decrease coverage by 0.05%.
The diff coverage is 22.72%.

@@            Coverage Diff             @@
##             main    #5964      +/-   ##
==========================================
- Coverage   68.29%   68.23%   -0.06%     
==========================================
  Files        1142     1142              
  Lines      242803   242821      +18     
  Branches    25385    25386       +1     
==========================================
- Hits       165825   165695     -130     
- Misses      70296    70433     +137     
- Partials     6682     6693      +11     
Flag Coverage Δ
Debug 68.23% <22.72%> (-0.06%) ⬇️
production 62.89% <22.72%> (-0.07%) ⬇️
test 88.61% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
.../Microsoft.ML.Data/Model/ModelOperationsCatalog.cs 53.60% <0.00%> (-1.78%) ⬇️
src/Microsoft.ML.TimeSeries/PredictionEngine.cs 76.96% <0.00%> (-3.63%) ⬇️
...c/Microsoft.ML.Data/Prediction/PredictionEngine.cs 87.83% <44.44%> (-4.81%) ⬇️
...rosoft.ML.Data/Model/PredictionEngineExtensions.cs 100.00% <100.00%> (ø)
...osoft.ML.KMeansClustering/KMeansPlusPlusTrainer.cs 83.60% <0.00%> (-7.16%) ⬇️
src/Microsoft.ML.Data/Training/TrainerUtils.cs 65.86% <0.00%> (-3.82%) ⬇️
...crosoft.ML.StandardTrainers/Standard/SdcaBinary.cs 85.06% <0.00%> (-3.08%) ⬇️
test/Microsoft.ML.AutoML.Tests/AutoFitTests.cs 87.07% <0.00%> (-1.97%) ⬇️
...crosoft.ML.StandardTrainers/Optimizer/Optimizer.cs 71.96% <0.00%> (-1.16%) ⬇️
...oft.ML.StandardTrainers/Standard/SdcaMulticlass.cs 91.12% <0.00%> (-1.03%) ⬇️
... and 6 more

@michaelgsharp michaelgsharp changed the title Prediction engine no longer disposes model. Prediction engine options Oct 9, 2021
{
/// <summary>
/// Options for the <see cref="PredictionEngine{TSrc, TDst}"/> as used in
/// [RandomizedPca(Options)](xref:Microsoft.ML.PcaCatalog.RandomizedPca(Microsoft.ML.AnomalyDetectionCatalog.AnomalyDetectionTrainers,Microsoft.ML.Trainers.RandomizedPcaTrainer.Options)).
Copy link
Member

@eerhardt eerhardt Oct 11, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a copy-paste error? #Resolved

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, resolved.

Copy link
Member

@eerhardt eerhardt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@michaelgsharp michaelgsharp merged commit 1dfccca into dotnet:main Oct 12, 2021
@michaelgsharp michaelgsharp deleted the predictionengine branch October 12, 2021 01:19
@ghost ghost locked as resolved and limited conversation to collaborators Mar 17, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

PredictionEngineBase Dispose() also calls ITransformer.Dispose()
3 participants