Drop multiples columns from the dataframe in python

Question

I want to drop multiple columns(around 800) from the dataframe using python. I have written below code:

  def corr_df(x, corr_val):


    # Creates Correlation Matrix and Instantiates
    corr_matrix = x.corr()
    iters = range(len(corr_matrix.columns) - 1)
    drop_cols = []
    df_drop=pd.DataFrame()
    cols=[]
    # Iterates through Correlation Matrix Table to find correlated columns
    for i in iters:
        for j in range(i):
            item = corr_matrix.iloc[j:(j+1), (i+1):(i+2)]
            col = item.columns
            row = item.index
            val = item.values
            if val >= corr_val:
                # Prints the correlated feature set and the corr val
                #print(col.values[0], "|", row.values[0], "|", round(val[0][0], 2))
                drop_cols.append(i)

    drops = sorted(set(drop_cols))[::-1]
    df_dropped=x.drop(drops,axis=1)
    # Drops the correlated columns
#     for i in drops:
#         col=(x.iloc[:, (i+1):(i+2)].columns.values.tolist())
#         print (col)
#         df_dropped=df.drop(col, axis=1)

        #cols.append()

    #print(df_dropped)
    return (df_dropped)

But this code is printing the dataframe have only one column dropped. Any comments or suggestions on this?

Thanks in advance

Use df = df.drop(lst, axis=1), where lst is your list of columns. Is there a reason why you need to perform this one at a time iteratively? — jpp
– jpp, Commented Feb 5, 2018 at 18:58
I can drop all of them once, but when I am doing the df.drop(drops,axis=1). Getting ValueError: labels [1069 1068 1067 ..., 3 2 1] not contained in axis. — user15051990
– user15051990, Commented Feb 5, 2018 at 19:03
Yup, But columns are there. I don't why it is throwing valueError. — user15051990
– user15051990, Commented Feb 5, 2018 at 19:05
@jp_data_analysis, I have added full code, please have a look. — user15051990
– user15051990, Commented Feb 5, 2018 at 19:06

jpp · Accepted Answer · 2018-02-05 19:10:27Z

1

Drop multiple columns by numerical index like this:

cols = [1069, 1068, 1067]

df = df.drop(df.columns[cols], axis=1)

answered Feb 5, 2018 at 19:10

jpp

166k37 gold badges301 silver badges363 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Drop multiples columns from the dataframe in python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related