L1 and L2 Regularization

L1 Regularization: Also known as Lasso (Least Absolute Shrinkage and Selection Operator) regularization. It adds the absolute value of coefficients as a penalty term to the loss function. The effect of L1 regularization is that it can lead to sparse models, where some coefficients can become exactly zero. This property makes it useful for feature selection in models with many features.
L2 Regularization: Also known as Ridge regularization. It adds the squared value of coefficients as a penalty term to the loss function. L2 regularization tends to distribute the error among all terms, leading to smaller and more robust coefficient estimates, but it does not necessarily set coefficients to zero. This makes it less useful for feature selection but better for improving model performance by reducing overfitting.

Naixian Zhang