I have df column with lists. Each looks like [1,2,3,4,'',6,7],[2,3,'',5,6]. I want to remove the '' in each row. I used
df[column].apply(lambda x: x.remove(''))
But it didn't work. Could some one help me? Thanks
ValueError: list.remove(x): x not in list
If you have an list , you can use filter on it to filter out the elements that you don't want to be in the list as follows -
my_list = [1,2,3,4,'',6,7]
my_list = list(filter(lambda x : x!='',my_list))
print(my_list)
OUTPUT :
[1, 2, 3, 4, 6, 7]
This can also be easily extended for an 2-d list using map. We will basically just apply the first function on every element of a 2-d list as follows -
my_list = [[1,2,3,4,'',6,7],[2,3,'',5,6]]
my_list = list(map(lambda sub_list:list(filter(lambda x : x!='',sub_list)),my_list))
print(my_list)
OUTPUT :
[[1, 2, 3, 4, 6, 7], [2, 3, 5, 6]]
Make an explicit filter on it: filter(lambda x: x != "", your_list) or use a list comprehension: [x for x in your_list if x != ""]. They work the same, just a matter of preference.
You don't want to filter out by a boolean method because then you'd accidentally get rid of 0s because they're "falsy" in python.
For your particular issue, you can simply put it in a try block, it throws an error when it comes across lists that don't have "" as an element for those you can
for i, row in df.iterrows():
row['column'] = list(filter(lambda x: x != "", row['column']))
If you just want ways to remove "" from list,
You can try,
new_list= [s for s in your_list if s != ""]
or
new_list = list(filter(lambda x: x != "", your_list))
also works
.map or .apply instead would make this the best option I think.i dont know how exactly u got the error. if your columns just has two elemens as mentioned, it should work as shown below.
import pandas as pd
df = pd.DataFrame({'column':[[1,2,3,4,'',6,7],[2,3,'',5,6]]})
df['column'].apply(lambda x:x.remove(''))
print(df)
Output
column
0 [1, 2, 3, 4, 6, 7]
1 [2, 3, 5, 6]
the problem may happened because u may have an element which have no '' in it like [2,3,5,6] so in that case this error may occur. so just recreating the error we can see the same error happening.
import pandas as pd
df = pd.DataFrame({'column':[[1,2,3,4,'',6,7],[2,3,'',5,6],[2,3,5,6]]})
df['column'].apply(lambda x:x.remove(''))
print(df)
Error Output
Traceback (most recent call last):
File "C:/Users/Deva/PycharmProjects/IITJ/ML/MLf3/extras/adaf.py", line 3, in <module>
df['column'].apply(lambda x:x.remove(''))
File "C:\Users\Deva\AppData\Local\Programs\Python\Python38\lib\site-packages\pandas\core\series.py", line 3848, in apply
mapped = lib.map_infer(values, f, convert=convert_dtype)
File "pandas\_libs\lib.pyx", line 2329, in pandas._libs.lib.map_infer
File "C:/Users/Deva/PycharmProjects/IITJ/ML/MLf3/extras/adaf.py", line 3, in <lambda>
df['column'].apply(lambda x:x.remove(''))
ValueError: list.remove(x): x not in list
Solution For above situation
To avoid that just introduce try except
import pandas as pd
df = pd.DataFrame({'column':[[1,2,3,4,'',6,7],[2,3,'',5,6],[2,3,5,6]]})
try:
df['column'].apply(lambda x:x.remove(''))
except:
pass
print(df)
Output
column
0 [1, 2, 3, 4, 6, 7]
1 [2, 3, 5, 6]
2 [2, 3, 5, 6]
