MENU
Edit
In Place Editing Advanced Editing

Lasso and Elastic Net for Sparse Signals

Estimates Lasso and Elastic-Net regression models on a manually generated sparse signal corrupted with an additive noise. Estimated coefficients are compared with the ground-truth.

1
2
3
4
5
6
print(__doc__)
 
import numpy as np
import matplotlib.pyplot as plt
 
from sklearn.metrics import r2_score

generate some sparse data to play with

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
np.random.seed(42)
 
n_samples, n_features = 50, 200
X = np.random.randn(n_samples, n_features)
coef = 3 * np.random.randn(n_features)
inds = np.arange(n_features)
np.random.shuffle(inds)
coef[inds[10:]] = 0  # sparsify coef
y = np.dot(X, coef)
 
# add noise
y += 0.01 * np.random.normal((n_samples,))
 
# Split data in train set and test set
n_samples = X.shape[0]
X_train, y_train = X[:n_samples / 2], y[:n_samples / 2]
X_test, y_test = X[n_samples / 2:], y[n_samples / 2:]

Lasso

1
2
3
4
5
6
7
8
9
from sklearn.linear_model import Lasso
 
alpha = 0.1
lasso = Lasso(alpha=alpha)
 
y_pred_lasso = lasso.fit(X_train, y_train).predict(X_test)
r2_score_lasso = r2_score(y_test, y_pred_lasso)
print(lasso)
print("r^2 on test data : %f" % r2_score_lasso)

Out:

1
2
3
4
  Lasso(alpha=0.1, copy_X=True, fit_intercept=True, max_iter=1000,
   normalize=False, positive=False, precompute=False, random_state=None,
   selection='cyclic', tol=0.0001, warm_start=False)
r^2 on test data : 0.384710

ElasticNet

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
from sklearn.linear_model import ElasticNet
 
enet = ElasticNet(alpha=alpha, l1_ratio=0.7)
 
y_pred_enet = enet.fit(X_train, y_train).predict(X_test)
r2_score_enet = r2_score(y_test, y_pred_enet)
print(enet)
print("r^2 on test data : %f" % r2_score_enet)
 
plt.plot(enet.coef_, color='lightgreen', linewidth=2,
         label='Elastic net coefficients')
plt.plot(lasso.coef_, color='gold', linewidth=2,
         label='Lasso coefficients')
plt.plot(coef, '--', color='navy', label='original coefficients')
plt.legend(loc='best')
plt.title("Lasso R^2: %f, Elastic Net R^2: %f"
          % (r2_score_lasso, r2_score_enet))
plt.show()

../../_images/sphx_glr_plot_lasso_and_elasticnet_001.png

Out:

1
2
3
4
  ElasticNet(alpha=0.1, copy_X=True, fit_intercept=True, l1_ratio=0.7,
      max_iter=1000, normalize=False, positive=False, precompute=False,
      random_state=None, selection='cyclic', tol=0.0001, warm_start=False)
r^2 on test data : 0.240176

Total running time of the script: (0 minutes 0.203 seconds)

Download Python source code: plot_lasso_and_elasticnet.py
Download IPython notebook: plot_lasso_and_elasticnet.ipynb
doc_scikit_learn
doc_scikit_learn
2025-01-10 15:47:30
Comments
Leave a Comment

Please login to continue.