Regularization Quiz (Hard)

What happens to the coefficients in Ridge Regression as the regularization parameter approaches zero?

In the context of regularization, under which circumstance would you prefer to use Elastic Net over LASSO and Ridge?

In the context of regularization, under which circumstance would you prefer to use LASSO over Ridge?

In which scenario would you prefer using Ridge Regression over LASSO?

How does increasing the regularization parameter in LASSO affect the bias and variance of the model?

What is a potential drawback of using a very strong LASSO regularization (very high lambda)?

What is the geometric interpretation of the regularization term in Ridge Regression when viewed as a constraint in the optimization problem?

What is the geometric interpretation of the regularization term in LASSO Regression when viewed as a constraint in the optimization problem?

In Elastic Net, the mixing parameter (denoted as “l1_ratio”) controls the trade-off between L1 and L2 regularization. What is the range of values for the mixing parameter? 

Consider a scenario where you are applying LASSO Regression to a dataset with highly correlated features. How does the introduction of L1 regularization affect the behavior of the model in terms of selecting features?