How to use multiple regex groups in pyspark?

Question

I want to insert a symbol between two regex groups.

My code is as follows:

df = spark.createDataFrame([('ab',)], ['str'])
df = df.select(
  concat(
    regexp_extract('str', r'(\w)(\w)', 1),  # extract the first group
    lit(' '),                               # add symbol
    regexp_extract('str', r'(\w)(\w)', 2)   # add the second group
  ).alias('d')).collect()
print(df)

Is there any better way?

mck · Accepted Answer · 2021-05-19 07:54:35Z

1

You can use regexp_replace with capture groups:

import pyspark.sql.functions as F

df.select(F.regexp_replace('str', r'(\w)(\w)', '$1 $2').alias('d')).show()
+---+
|  d|
+---+
|a b|
+---+

answered May 19, 2021 at 7:54

mck

42.7k13 gold badges44 silver badges62 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Andrey Over a year ago

thanks. I tried regexp_replace('str', r'(\w)(\w)', '\1 \2') with standard re group markers and it didn't work. Is use of $ in pyspark documented somewhere ?

Collectives™ on Stack Overflow

How to use multiple regex groups in pyspark?

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related