Python Fitting Linear Regression using Greedy Feature Selection

Question

I'm trying to fit a linear regression model using a greedy feature selection algorithm. To be a bit more specific, I have four sets of data:

X_dev, y_dev, X_test, y_test, the first two being the features and labels for the training set and the latter two for the test set. The size of the matrices are (900, 126), (900, ), (100, 126), and (100, ), respectively.

What I mean by "greedy feature selection" is that I would first fit 126 models using one feature each from the X_dev set, choose the best one, then run models using the first one and each of the remaining 125 models. The selection continues until I have obtained 100 of the features that perform best among the original 126.

The problem I'm facing is regarding implementation in Python. The code that I have is for fitting a single feature first:

lin_reg.fit(X_dev[:, 0].reshape(-1, 1), y_dev)
lin_pred = lin_reg.predict(X_test)

Because the dimensions don't match ((100, 126) and (1, )) I'm getting a dimension mismatch error.

How should I fix this? I'm trying to predict how the model performs when using the single feature.

Thank you.

Also, there is undoubtedly an implementation of this already built for you in scikit or some other library. Find it and save yourself the hassle — John R
– John R, Commented Nov 1, 2018 at 4:18
Unfortunately, this is basically the entire code that I have atm. The rest are the usual import statements for NumPy and sklearn with the np.load statements for the data. I'll take your advice though and look for an implementation. — Sean
– Sean, Commented Nov 1, 2018 at 4:44

John R · Accepted Answer · 2018-11-01 04:51:57Z

1

Apply the same transformation to X_test

lin_reg.fit(X_dev[:, 0].reshape(-1, 1), y_dev)
lin_pred = lin_reg.predict(X_test[:, 0].reshape(-1, 1))

I also don’t think the reshape is necessary.

lin_reg.fit(X_dev[:, 0], y_dev)
lin_pred = lin_reg.predict(X_test[:, 0])

Should work as well

answered Nov 1, 2018 at 4:51

John R

1,51611 silver badges19 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Sean Over a year ago

Thanks @John H your suggestion helped me solve it. The second method you proposed is the one I initially tried, but got the error message ValueError: Expected 2D array, got 1D array instead.

Collectives™ on Stack Overflow

Python Fitting Linear Regression using Greedy Feature Selection

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related