Regularization
Last modified on October 09, 2021
Links to “Regularization”
Bias-variance trade off: (Learning a generative model (Maximum Likelihood) > Bias-variance trade off:)
Bias limitation: If the hypothesis space of functions is very limited, we might not be able to represent the data distribution.
Variance limitation: If the hypothesis space is too expressive, it will overfit to the data.
How to prevent overfitting? Prefer “simpler” models (Occam’s razor.) Regularization in the objective function. Evaluate on validation set while training.