Hyperparameters are not the same as Model Parameters

20 Apr, 2022 • 3 min read

These important terminologies could easily be mistaken for each other, but they must not and should not be. Once you take a look at the examples below, you would instantly get the difference between these two similar-sounding terms in machine learning.

We don't meow the same way.

What's an hyperparameter?

Consider the implementation of a neural network algorithm, there are certain parameters that can be specified before model training, such as number of epochs or number of clusters that should be identified by the model. These parameters are referred to as hyperparameters.

They are not changed during model training, and are set manually by you, the developer.

What then is a model parameter?

Model parameters on the other hand are the core parameters that are required for making predictions. Just like the neural networks example above, it's model parameters are the weights and biases that are constantly updated during training.

These parameters are estimated by optimization algorithms such as gradient descent or normal equation.

Instances of hyperparameters

As mentioned earlier, you manually set your hyperparameters before the learning process begins. This process is called "hyperparameter tuning" and is one of the key concepts to know in machine learning.

Consider a k-means clustering algorithm, an hyperparameter would be? Yes, the number of clusters you want your model to create! Again, in neural networks, you can specify the number of layers and the number of neurons in every layer.

Instances of model parameters

What parameters are updated during training in a neural network? Weights and biases. These are parameters in a neural network algorithm. For a Linear Regression algorithm, the parameters are slope and intercept.

Conclusion

You cannot set the parameters of a model (do not mistake this for randomly setting initial values for weights and biases in a neural network). You can however determine the hyperparameters of a model.

These hyperparameters influence the final results of a model parameter. A large number of iterations of gradient descent will create parameters different from a small number of iterations.

And finally, hyperparamters will estimate how optimized the model training is while model parameters will estimate how well the model will perform when put to the test to predict new unseen data.

Page updated

Google Sites

Report abuse