get unique strings inside the list of each row python

Question

I have a dataframe

                                dt  ... contains
0        2021-03-19 14:59:49+00:00  ...     [up]
1        2021-03-19 14:59:51+00:00  ...       []
2        2021-03-19 14:59:51+00:00  ...       []
3        2021-03-19 14:59:51+00:00  ...       []
4        2021-03-19 14:59:52+00:00  ...       []
                            ...  ...      ...
1624153  2021-07-30 08:53:30+00:00  ...    [buy, buy, buy]

for the column contains I want to have unique strings inside of each list like this

                                dt  ... contains
0        2021-03-19 14:59:49+00:00  ...     [up]
1        2021-03-19 14:59:51+00:00  ...       []
2        2021-03-19 14:59:51+00:00  ...       []
3        2021-03-19 14:59:51+00:00  ...       []
4        2021-03-19 14:59:52+00:00  ...       []
                            ...  ...      ...
1624153  2021-07-30 08:53:30+00:00  ...    [buy]

Thank you

Does the order matter? And are the values python list or the string representation of the list? — ThePyGuy
– ThePyGuy, Commented Aug 5, 2021 at 10:19
order doesn't matter I tried using set but it didn't really worked — YanRemes
– YanRemes, Commented Aug 5, 2021 at 10:22

ThePyGuy · Accepted Answer · 2021-08-05 10:22:33Z

1

If the order of the values in the list doesn't matter, just convert them to set then back to list:

df['contains']=df['contains'].apply(set).apply(list)

# You can also use a single apply with lambda
# df['contains']=df['contains'].apply(lambda x: list(set(x)))

If the order matters, you can create a helper function to return the list of unique values only, then apply it on the column:

def getUnique(x):
    new = []
    for i in x:
        if i not in new:
            new.append(i)
    return new

df['contains'] = df['contains'].apply(getUnique)

answered Aug 5, 2021 at 10:22

ThePyGuy

18.5k5 gold badges24 silver badges55 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

YanRemes Over a year ago

also, how would you count how many words are in each list?

ThePyGuy Over a year ago

Unique count or just the count?

YanRemes Over a year ago

now I have a table with unique words and I just want to count how many words are in each list for each row

ThePyGuy Over a year ago

Then just use df['contains'].apply(len)

Collectives™ on Stack Overflow

get unique strings inside the list of each row python

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related