Get index and columns based on conditional

Question

Having a DF as the example below, how can i get list with the index and column name of the values that arent nan?

                  es  ms        hs
subdist_id                        
1                NaN NaN       NaN
2                NaN NaN       NaN
3          -0.218066 NaN -0.309002
4                NaN NaN       NaN
5                NaN NaN       NaN
6                NaN NaN       NaN
7                NaN NaN       NaN
9                NaN NaN       NaN
10               NaN NaN       NaN
11         -0.385217 NaN       NaN

so i can get a list [[3,'es]', [3,'hs'], [11,'es']]

jezrael · Accepted Answer · 2019-03-29 21:01:32Z

3

Use DataFrame.stack for remove NaNs with convert MultiIndex to list of tuples and then convert it to list of lists:

L = list(map(list, df.stack().index.tolist()))
print (L)
[[3, 'es'], [3, 'hs'], [11, 'es']]

Alternative:

L = [list(x) for x in df.stack().index.tolist()]
print (L)
[[3, 'es'], [3, 'hs'], [11, 'es']]

If need performance,get indices by condition by DataFrame.isna and numpy.where and get values by zip with indexing for values of columns and index:

i,c = np.where(df.notna())
L = list(map(list, zip(df.index[i],df.columns[c])))
print (L)
[[3, 'es'], [3, 'hs'], [11, 'es']]

i,c = np.where(df.notna())
L = [list(x) for x in zip(df.index[i],df.columns[c])]

edited Mar 29, 2019 at 21:01

answered Mar 29, 2019 at 20:22

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Get index and columns based on conditional

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related