Pandas select columns by regex and change their values by if, else

Question

I have Pandas data frame like this:

   a      b1         b2         b3       b4       c1      c2       c3         c4
   a1     0.10       0.0        0.21     0.0      0.03    0.10     0.04      0.0

How can I change it to the following:

   a      b1         b2         b3       b4       c1      c2       c3         c4
   a1     1          0           1       0        1       0        1          0

So, I want to select b* and c* columns and change any non-zero value into 1 and any zero value into 0. So, first selecting columns by regex then apply if-else rule there. It is also worth noting that all b*, c* columns are string (obj) types.

How can I do this?

Andrej Kesely · Accepted Answer · 2020-10-20 17:22:33Z

3

Regex is not necessary, use str.startswith instead:

filter_col = [col for col in df if col.startswith('b') or col.startswith('c')]
df[filter_col] = (df[filter_col] > 0).astype(int)
print(df)

Prints:

    a  b1  b2  b3  b4  c1  c2  c3  c4
0  a1   1   0   1   0   1   1   1   0

Edit: If your "numbers" are strings originally, you can do:

filter_col = [col for col in df if col.startswith('b') or col.startswith('c')]
df[filter_col] = (df[filter_col].astype(float) > 0).astype(int)
# if you want keep them as strings after computation:
# df[filter_col] = (df[filter_col].astype(float) > 0).astype(int).astype(str)
print(df)

edited Oct 20, 2020 at 17:22

answered Oct 20, 2020 at 17:14

Andrej Kesely

196k15 gold badges60 silver badges105 bronze badges

Sign up to request clarification or add additional context in comments.

7 Comments

learner Over a year ago

Hi @Andrej Kesley, I want to preserve the original data types of columns as string.

Paul H Over a year ago

@SumitSidana why are you storing numbers as strings? what's the broader context here?

Andrej Kesely Over a year ago

@SumitSidana Do you mean that the number 0 and 1 should be string and not integers?

learner Over a year ago

@AndrejKeselyI get TypeError: '>' not supported between instances of 'str' and 'int' for your solution. It is worth noting that all filter_col are str type.

Paul H Over a year ago

@SumitSidana I would store numbers as numbers for as long as possible in your analysis pipeline and convert to strings as the very last step

|

Quang Hoang · Accepted Answer · 2020-10-20 17:18:33Z

2

Another option is str.match:

mask = df.columns.str.match('^(b|c)')
df.loc[:, mask] = np.where(df.loc[:,mask].astype(float)==0, '0', '1')

answered Oct 20, 2020 at 17:18

Quang Hoang

151k11 gold badges64 silver badges86 bronze badges

Collectives™ on Stack Overflow

Pandas select columns by regex and change their values by if, else

2 Answers 2

7 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

7 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related