Create pandas dataframe out of another dataframe fast

Question

I have got a dataFrame which looks like this:

index | in | out | time
   7  |  8 |  8  |  232
  11  |  3 |  0  |    0
  79  |  0 |  8  |   12

And I want to create a DataFrame out of this one, where every non-zero in/out value is set to 1 (they are all positive). Time and index should be the same:

index | in | out | time
   7  |  1 |  1  |  232
  11  |  1 |  0  |    0
  79  |  0 |  1  |   12

I think there should be a faster way, than how I am doing this:

df2 = pd.DataFrame({"index":[], "in":[], "out":[], "time":[]})
for index, row in df.iterrows():
    if row["in"] == 0:
        in_val = 0
    else:
        in_val = 1
    if row["out"] == 0: 
        out_val = 0
    else:
        out_val = 1
    time = row["time"]
    df2 = df2.append(pd.DataFrame({"index":[index], "in":[in_val], "out":[out_val], "time":[time]}), sort=False)

Can I use some lambda function or something like a list comprehension to convert the dataframe faster?

use np.where to change values other than 1 to 1

moys
– moys

2019-09-10 06:27:31 +00:00
Commented Sep 10, 2019 at 6:27 — moys
– moys, Commented Sep 10, 2019 at 6:27

jezrael · Accepted Answer · 2019-09-10 06:27:47Z

4

Use numpy.where with columns with lists:

cols = ['in','out']
df[cols] = np.where(df[cols].eq(0), 0, 1)

Or cast boolean mask for not equal to integers:

df[cols] = df[cols].ne(0).astype(int)

If no negative values use DataFrame.clip:

df[cols] = df[cols].clip(upper=1)
print (df)
   index  in  out  time
0      7   1    1   232
1     11   1    0     0
2     79   0    1    12

answered Sep 10, 2019 at 6:27

jezrael

868k103 gold badges1.4k silver badges1.3k bronze badges

Sign up to request clarification or add additional context in comments.

Comments

anky · Accepted Answer · 2019-09-10 06:30:14Z

1

Alternatively you can use astype to convert to boolean and multiply with 1:

cols=['in','out']
df[cols]=df[cols].astype(bool)*1

   index  in  out  time
0      7   1    1   232
1     11   1    0     0
2     79   0    1    12

answered Sep 10, 2019 at 6:30

anky

75.3k11 gold badges46 silver badges76 bronze badges

Comments

tawab_shakeel · Accepted Answer · 2019-09-10 06:30:07Z

0

use np.where()

df=pd.DataFrame(data={"in":[8,3,0],
                  "out":[8,0,8],
                  "time":[232,0,12]})

df[['in','out']] = np.where(df[['in','out']] == 0, 0, 1)

   in   out time
0   1   1   232
1   1   0   0
2   0   1   12

answered Sep 10, 2019 at 6:30

tawab_shakeel

3,75912 silver badges26 bronze badges

Comments

Sreeram TP · Accepted Answer · 2019-09-10 06:30:55Z

0

So you have a dataframe like this,

    index   in  out     time
0   7   8   8   232
1   11  3   0   0
2   79  0   8   12

Use np.where to get the desired result like this,

df['in'] = np.where(df['in'] > 0, 1, 0)
df['out' = np.where(df['out'] > 0, 1, 0)

answered Sep 10, 2019 at 6:30

Sreeram TP

12k8 gold badges65 silver badges121 bronze badges

Comments

Sandun Menaka Jayawardhena · Accepted Answer · 2019-09-10 06:38:36Z

0

You can try

df['in'] = [1 if i>0 else 0 for i in list(df['in'])]

answered Sep 10, 2019 at 6:38

Sandun Menaka Jayawardhena

1416 bronze badges

Collectives™ on Stack Overflow

Create pandas dataframe out of another dataframe fast

5 Answers 5

Comments

Comments

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

Comments

Comments

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related