How to replace values by None if string contains character pattern?

Question

I want to replace every string in my pandas df column departments with None if it contains a )

         departments   var1   var1.1
   1      transport     aa      uu
   2      industry)     bb      tt
   3      aviation)     cc      tt

how the dataset should look like

         departments   var1    var2
   1      transport     aa      uu
   2      None          bb      tt
   3      None          cc      tt

A similar solution is here: Replacing regex pattern with another string works, but replacing with NONE replaces all values

How can i transform it to base python as i dont use spark?

df.withColumn("departments", when(col("departments").rlike("\)"), None)
          .otherwise(col("departments"))
      )

so is the above df shown a pandas df or a spark df?

Sammy J
– Sammy J

2021-06-07 17:18:38 +00:00
Commented Jun 7, 2021 at 17:18 — Sammy J
– Sammy J, Commented Jun 7, 2021 at 17:18
my df is a pandas df

id345678
– id345678

2021-06-07 17:27:56 +00:00
Commented Jun 7, 2021 at 17:27 — id345678
– id345678, Commented Jun 7, 2021 at 17:27

RavinderSingh13 · Accepted Answer · 2021-06-07 17:28:54Z

4

With your shown samples, please try following. You could use str.contains function to find out whatever values in departments column has ) then using .loc with respect to values which we got in m variable setting None to those values.

m = df['departments'].str.contains('\)', na=False)
df.loc[m,'departments'] = None

edited Jun 7, 2021 at 17:28

answered Jun 7, 2021 at 17:20

RavinderSingh13

135k14 gold badges61 silver badges100 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

id345678 Over a year ago

ValueError: Cannot mask with non-boolean array containing NA / NaN values. The missing values will need to be explicitly filled with True or False prior to using the array as a mask.

RavinderSingh13 Over a year ago

@id345678, sure, I have edited my answer, kindly take a look to it and let me know how it goes then.

Collectives™ on Stack Overflow

How to replace values by None if string contains character pattern?

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related