Count elements in pandas dataframe columns

Question

Any suggestions for elegant ways to make the following counts for each column in my dataframe:

Count of elements in the column which are numeric (any value)
Count of elements in the column which are numeric and nonzero

Here's some code to generate a dataframe like mine (but much smaller):

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
#from matplotlib import style
import seaborn as sns

df = pd.DataFrame(data=np.random.random_integers(low=0,high=10,size=(50,1)),
                  columns=['all']
                  )

# set up the grouping
g = df.groupby(pd.cut(df['all'],[-1,2,4,6,8,10]))
for name, data in g.groups.items():
    df[name] = df.loc[data]['all']

This produces a dataframe that looks like the following:

So you expected out like beyond ?

BENY
– BENY

2017-11-03 00:46:33 +00:00
Commented Nov 3, 2017 at 0:46 — BENY
– BENY, Commented Nov 3, 2017 at 0:46

Andy Hayden · Accepted Answer · 2017-11-03 01:06:29Z

1

You can do this without the apply:

In [11]: df.notnull().sum()
Out[11]:
all        50
(-1, 2]    12
(2, 4]     12
(4, 6]      7
(6, 8]     10
(8, 10]     9
dtype: int64

In [12]: (df.notnull() & (df != 0)).sum()
Out[12]:
all        46
(-1, 2]     8
(2, 4]     12
(4, 6]      7
(6, 8]     10
(8, 10]     9
dtype: int64

Note: This works as counting True values is the same as taking their sum.

answered Nov 3, 2017 at 1:06

Andy Hayden

378k110 gold badges640 silver badges546 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

BENY · Accepted Answer · 2017-11-03 00:56:41Z

0

Count of elements in the column which are numeric (any value)

df.apply(lambda x : x.notnull().sum())
Out[262]: 
all        50
(8, 10]     9
(-1, 2]    12
(4, 6]      9
(2, 4]     12
(6, 8]      8
dtype: int64

Count of elements in the column which are numeric and nonzero

df.mask(df==0).apply(lambda x : x.notnull().sum())
Out[263]: 
all        48
(8, 10]     9
(-1, 2]    10
(4, 6]      9
(2, 4]     12
(6, 8]      8
dtype: int64

answered Nov 3, 2017 at 0:56

BENY

324k22 gold badges176 silver badges250 bronze badges

Collectives™ on Stack Overflow

Count elements in pandas dataframe columns

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related