Python Regression Variable Selection

Question

I have a basic linear regression with 80 numerical variables (no classification variables). Training set has 1600 rows, testing 700.

I would like a python package that iterates through all column combinations to find the best custom score function or an out of the box score funtion like AIC. OR If that doesnt exist, what do people here use for variable selection? I know R has some packages like this but dont want deal with Rpy2

I have no preference if the LM requires scikit learn, numpy, pandas, statsmodels, or other.

scikit-learn.org/stable/modules/feature_selection.html ?

John Zwinck
– John Zwinck

2014-11-07 12:45:27 +00:00
Commented Nov 7, 2014 at 12:45 — John Zwinck
– John Zwinck, Commented Nov 7, 2014 at 12:45

Takahiro Yoshizawa · Accepted Answer · 2016-03-11 20:20:45Z

4

I can suggest an answer that using the Least Absolute Shrinkage and Selection Operator(Lasso). I didn't use in a situation like you, that you have to deal with so many data.

http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.Lasso.html

I often write a code to do linear regression with statsmodels like below,

import statsmodels.api as sm

model = sm.OLS()
results = model.fit(train_X,train_Y)

If I want to do Lasso regression, I write a code like below,

from sklearn import linear_model

model = linear_model.Lasso(alpha=1.0(default))
results = model.fit(train_X,train_Y)

You have to decide appropriate alpha between 0.0 and 1.0. The parameter is determined by how you don't accept the error.

Try this.

edited Mar 11, 2016 at 20:20

answered Mar 11, 2016 at 19:20

Takahiro Yoshizawa

1111 gold badge1 silver badge9 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Mogsdad Over a year ago

This may theoretically answer the question, but it would be best to include the essential parts of the answer here for future users, and provide the link for reference. Link-dominated answers can become invalid through link rot.

Collectives™ on Stack Overflow

Python Regression Variable Selection

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related