0

I am looking into the selected features in hybrid feature selection which consists of embedded feature selection and wrapper selection. So, I get the features with their feature importance and then run the wrapper selection using the selected features in the embedded selection and get the features with best model accuracy.

I got the bar chart from the embedded selection and now I want to just colour bars that for features selected in wrapper selection. How can I approach this? See my following code,

############################################# Hybrid Feature Selection Methodology #####################################
                                #################### Embedded Method ########################
# perform permutation importance
results = permutation_importance(knn, X_train, y_train, scoring='accuracy')
# get importance
importance = results.importances_mean
print(importance)
# summarize feature importance
for i,v in enumerate(importance):
    print('Feature: %0d, Score: %.5f' % (i,v))
# plot feature importance
plt.bar([x for x in range(len(importance))], importance)
plt.title('Permutation Feature Importance with KNN')
plt.xlabel('Features')
plt.ylabel('Feature Importance')
plt.show()

                                #################### Wrapper Method ########################
efs = EFS(knn, min_features=1, max_features=len(X_train_knn.columns), scoring='accuracy', print_progress=True, cv=2)

# fit the object to the training data.
efs = efs.fit(X_train_knn, y_train)
print('\n')
print('Best accuracy score: ', efs.best_score_ * 100)
print('Best subset (indices):', efs.best_idx_)
print('Best subset (corresponding names):', efs.best_feature_names_)

# transform our data to the newly selected features.
optimum_number_features = list(efs.best_idx_)
optimum_number_features_knn = list(efs.best_feature_names_)
7
  • if you're asking about coloring bars in a bar graph, remove all of the bits about the machine learning model. it only distracts from your question. stackoverflow.com/help/minimal-reproducible-example Commented Jun 10, 2020 at 0:03
  • @PaulH how about now? Commented Jun 10, 2020 at 1:04
  • Did you read the link I posted? Commented Jun 10, 2020 at 1:17
  • @PaulH I did. I edited the code to remove the unwanted parts Commented Jun 10, 2020 at 1:31
  • When I run your code, it says, "permutation_importance is not defined". So it's not really reproducible, is it? If you're asking about a bar plot, mock up (hard-code) some very simple data as lists or numpy arrays. Commented Jun 10, 2020 at 14:37

1 Answer 1

1

A Minimal, Reproducible Example partly means that everyone can execute your code and get result.

import matplotlib.pyplot as plt

# importance_list = list(zip(feature_name_list, results.importances_mean))
importance_list = [('quiz', 0.4080183920815765), ('time', 0.1779846287534165), ('hm', 0.1392329389521148), ('submitNum', 0.09889260035850235), ('class', 0.09379925836350246), ('post', 0.049803191453511066), ('startTime', 0.03226899003737626)]

plt.figure()

colors = ['b' for i in importance_list]

# selected_list is what your wrapper function returns
selected_list = ['quiz', 'time', 'hm']

for i, v in enumerate(importance_list):
    if v[0] in selected_list:
        colors[i] = 'r'

plt.bar([i[0] for i in importance_list], [i[1] for i in importance_list], color=colors)

plt.title('Permutation Feature Importance with KNN')
plt.xlabel('Features')
plt.ylabel('Feature Importance')

plt.show()

enter image description here

As for the zip function and importances_mean atrribute, I test it with examples from sklearn.inspection.permutation_importance.

from sklearn.linear_model import LogisticRegression
from sklearn.inspection import permutation_importance

X = [[1, 9, 9],[1, 9, 9],[1, 9, 9],
     [0, 9, 9],[0, 9, 9],[0, 9, 9]]
y = [1, 1, 1, 0, 0, 0]

clf = LogisticRegression().fit(X, y)

result = permutation_importance(clf, X, y, n_repeats=10, random_state=0)

list(zip(['a', 'b', 'c'], result.importances_mean))

# Result:
# [('a', 0.4666666666666666), ('b', 0.0), ('c', 0.0)]
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.