Model Selection Techniques
Once we’ve evaluated our models using cross-validation, the next step is selecting the best one for deployment. Model selection techniques help us compare different algorithms and configurations in a statistically robust way. Although we won’t cover it here because, frankly, it would be quite difficult to do so with all the different permutations of factors that exist, in the real world we’d also have a variety of business rules that would impact our decision. These are typically tied to monetary metrics around costs and savings incurred by utilizing a model versus using another technique. In this recipe, we’ll use grid search and randomized search to perform hyperparameter tuning and select the optimal model based on cross-validation scores.
Getting ready
We’ll use a classification task and compare different regularization strengths for logistic regression using both exhaustive and randomized search strategies.
Load the...
