Python DataFrame: Find duplicates in same row?

Question

I have a df with columns - section,classA,classB,classC.I'm trying to find duplicates values in a row.

df

                section       classA          classB          classC      

            0       A          paul            john             paul
            1       B          john            mark             tony
            2       C          leo             leo              leo
            3       D          tony            tony             mark
            4       E          paul
            5       F                          mark             mark

Final df

                section       classA          classB          classC        duplicate

            0       A          paul            john             paul          True
            1       B          john            mark             tony          False
            2       C          leo             leo              leo           True
            3       D          tony            tony             mark          True
            4       E          paul                                           False
            5       F                          mark             mark          True
            6       G                                                         False

I tried comparing each row, How to handle if row is empty?

Andrej Kesely · Accepted Answer · 2021-05-05 16:31:08Z

1

If the empty cells are empty strings (""), you can use set():

df["duplicate"] = df.apply(
    lambda x: len(set(x[x != ""])) != len(x[x != ""]), axis=1
)

print(df)

Prints:

  section classA classB classC  duplicate
0       A   paul   john   paul       True
1       B   john   mark   tony      False
2       C    leo    leo    leo       True
3       D   tony   tony   mark       True
4       E   paul                    False
5       F   mark   mark              True
6       G                           False

answered May 5, 2021 at 16:31

Andrej Kesely

196k15 gold badges60 silver badges105 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user15590480 Over a year ago

Should i replace NaN to ("") - empty strings?

Andrej Kesely Over a year ago

@user15590480 If they are NaN or None, you can use this lambda: lambda x: len(set(x[x.notna()])) != len(x[x.notna()])

Collectives™ on Stack Overflow

Python DataFrame: Find duplicates in same row?

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related