Pandas DataFrame Column of Lists: Remove a Specific Value

Question

I have a Pandas DataFrame with a column of lists. I want create a new column that is comprised of these same lists, minus one specific element:

[In]:
key1 = 'site channel fiscal_week'.split()
key2 = 'site dude fiscal_week'.split()
key3 = 'site eng fiscal_week'.split()

keys = pd.DataFrame({'key': [1,2,3],
                     'dims': [key1,key2,key3]})

keys

[Out]:
                         dims  key
[site, channel, fiscal_week]    1
[site, dude, fiscal_week]       2
[site, eng, fiscal_week]        3

Here is my approach that failed:

keys['reduced_dims'] = keys['dims'].remove('fiscal_week')

I need to be able to remove a specific element, not pop() off the last element.

Desired output:

[Out]:
                        dims  key  reduced_dims
[site, channel, fiscal_week]    1  [site, channel]
[site, dude, fiscal_week]       2  [site, dude]
[site, eng, fiscal_week]        3  [site, eng]

Mikhail Stepanov · Accepted Answer · 2018-08-21 19:12:35Z

3

keys['dims'] is a pd.Series, not list, and there's no list.remove() method. You should use pd.Series.apply() method, which applies some function to the values in each row.

keys['reduced_dims'] = keys['dims'].apply(
    lambda row: [val for val in row if val != 'fiscal_week']
)

keys['reduced_dims']

Out:
0    [site, channel]
1       [site, dude]
2        [site, eng]
Name: reduced_dims, dtype: object

And you can't use just list.remove() function instead of list comprehension,

lambda row: [val for val in row if val != 'fiscal_week']

because list.remove() returns None and you will get such series:

keys['reduced_dims'] = keys['dims'].apply(lambda x: x.remove('fiscal_week'))
keys['reduced_dims']

Out:
0    None
1    None
2    None
Name: reduced_dims, dtype: object

answered Aug 21, 2018 at 19:12

Mikhail Stepanov

3,8003 gold badges27 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Scott Boston Over a year ago

What is interesting about the last because remove acts "inplace" if you don't re-assign back to keys['reduced_dims'], your column series is still mutated and this will achieve the desired results.

NLR Over a year ago

I hadn't thought of applying list comprehension, but it makes perfect sense. Thanks!

SpghttCd · Accepted Answer · 2018-08-21 19:01:25Z

0

You could try

def rem_fw(lst):
    lst.remove('fiscal_week')
    return lst

keys['reduced_dims'] = keys['dims'].apply(rem_fw)

answered Aug 21, 2018 at 19:01

SpghttCd

10.9k2 gold badges23 silver badges28 bronze badges

Collectives™ on Stack Overflow

Pandas DataFrame Column of Lists: Remove a Specific Value

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related