Model hyper-parameters are usually chosen through a grid search procedure, in which a model is trained separately on every combination of hyper-parameters in the grid, and an error metric is stored after the model is evaluated on each setting combination. The setting that results in the lowest cross-validation error is usually considered the optimal combination. It is important to search enough combinations of the grid of hyperparameters to be able to see which settings are resulting in both under and overfitting, and that the optimal setting for each hyperparameter can be found within the range searched.
How are model hyper-parameters generally selected?
Help us improve this post by suggesting in comments below:
– modifications to the text, and infographics
– video resources that offer clear explanations for this question
– code snippets and case studies relevant to this concept
– online blogs, and research publications that are a “must read” on this topic
Partner Ad
