-
Notifications
You must be signed in to change notification settings - Fork 1.9k
image classification needs cancel #4632
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Hi @LittleLittleCloud , so it's still not clear to me what is being asked, and so if it would be a quick fix or a major feature request. Can you please share code (and perhaps a dataset) where you "cancel and switch to another scenario" (as mentioned in the issue you've linked to)? I would like to know how you're currently cancelling the training. Particularly, it's strange that you mention that there's a lot of memory being used and that's not released after cancelling. Perhaps you are pointing to a memory leak? Thanks. |
It's not a memory leak, the thing is our program allows user to cancel training after it start, But we can't stop an image training after it starts because current Image Classification API doesn't support cancellation token. The way we use Image Classification is through AutoML, there's a trainer in MulticlassificationExperiment , and that's how it be called |
This should be an easy fix. Please refer to the below PRs where we have added cancellation mechanism in other trainers. You can invoke cancellation via mlcontext cancel function that is exposed in experimental nuget. First, someone needs go add cancellation checkpoints, test, verify with model builder, commit, ship. https://github.com/dotnet/machinelearning/pulls?utf8=%E2%9C%93&q=is%3Apr+is%3Aclosed+cancellation++ |
Would doing what @codemzs has just said (to expose cancelation via mlContext.CancelExecution()) be enough for you, @LittleLittleCloud ? |
If we can cancel using CancelExecution() that also works. ( As long as there's a way to cancel training) |
Yep, we will need to add a check point in the training loop, it’s just few lines of change plus testing. |
Ok, so I will work on this. Thanks for your suggestions, @codemzs ! |
Hi, is there any plan to add cancel support for image classification API so that we can cancel an image classification training by passing in a cancellation token?
related issue
The text was updated successfully, but these errors were encountered: