I am looking into the selected features in hybrid feature selection which consists of embedded feature selection and wrapper selection. So, I get the features with their feature importance and then run the wrapper selection using the selected features in the embedded selection and get the features with best model accuracy.
I got the bar chart from the embedded selection and now I want to just colour bars that for features selected in wrapper selection. How can I approach this? See my following code,
############################################# Hybrid Feature Selection Methodology #####################################
#################### Embedded Method ########################
# perform permutation importance
results = permutation_importance(knn, X_train, y_train, scoring='accuracy')
# get importance
importance = results.importances_mean
print(importance)
# summarize feature importance
for i,v in enumerate(importance):
print('Feature: %0d, Score: %.5f' % (i,v))
# plot feature importance
plt.bar([x for x in range(len(importance))], importance)
plt.title('Permutation Feature Importance with KNN')
plt.xlabel('Features')
plt.ylabel('Feature Importance')
plt.show()
#################### Wrapper Method ########################
efs = EFS(knn, min_features=1, max_features=len(X_train_knn.columns), scoring='accuracy', print_progress=True, cv=2)
# fit the object to the training data.
efs = efs.fit(X_train_knn, y_train)
print('\n')
print('Best accuracy score: ', efs.best_score_ * 100)
print('Best subset (indices):', efs.best_idx_)
print('Best subset (corresponding names):', efs.best_feature_names_)
# transform our data to the newly selected features.
optimum_number_features = list(efs.best_idx_)
optimum_number_features_knn = list(efs.best_feature_names_)
