How should I remove nan values from a dataframe in python?

Question

I've got an excel file and I created lists from its columns. The problem is the rows of the columns is not equal. Therefore, I have multiple 'nan' values at ends of the lists. I tried to delete them with dropna() method but there are still the 'nan' values. Here is my code:

import pandas as pd

excel_name = r'file_name.xlsx'
df = pd.read_excel(excel_name, engine='openpyxl')
df.dropna()

clomun_1 = list(df['clomun1'])
clomun_2 = list(df['clomun2'])
clomun_3 = list(df['clomun3'])
print(clomun_1)
print(clomun_2)
print(clomun_3)

output:

clomun_1 = ['value1', 'value2', 'value3', 'value4', 'nan', 'nan', 'nan', 'nan']
clomun_2 = ['value1', 'value2', 'value3', 'value4', 'value5', 'value6', 'nan', 'nan']
clomun_3 = ['value1', 'value2', 'nan', 'nan', 'nan', 'nan', 'nan', 'nan']

I want to keep only values. I must delete "nan" elements.

Does this answer your question? How to convert 'NaN' strings in a pandas Series to null values for dropna? — Amit Singh
– Amit Singh, Commented Jun 2, 2021 at 19:13
I think you're reading literal "nan" strings from the spreadsheet. A numerical nan wouldn't have quotes around them. That's probably why dropna didn't drop them. — Ben Y
– Ben Y, Commented Jun 2, 2021 at 19:17

Gun · Accepted Answer · 2021-06-02 19:48:10Z

3

Try this:

df = pd.read_excel(excel_name, engine='openpyxl', na_values=['nan']) #add na_values

clomun_1 = df['clomun1'].dropna().tolist()

print(clomun_1)

['value1', 'value2', 'value3', 'value4']

edited Jun 2, 2021 at 19:48

answered Jun 2, 2021 at 19:23

Gun

5861 gold badge6 silver badges25 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Nirevezs Over a year ago

returns : [True, True, True, True]

Nirevezs Over a year ago

That's actually works. Thank you. You're the man.

Abishek VK · Accepted Answer · 2021-06-02 19:22:04Z

0

You can use a lambda function to achieve this.

clomun_1_new= [x for x in clomun_1 if x!='nan']

repeat the same for other two lists.

answered Jun 2, 2021 at 19:22

Abishek VK

5243 silver badges12 bronze badges

2 Comments

Nirevezs Over a year ago

Not working. 'nan ' values stille there after print(clomun_1_new)

Abishek VK Over a year ago

It is working for me. Try changing the line to: clomun_1_new= [x for x in clomun_1 if str(x) != 'nan']

Collectives™ on Stack Overflow

How should I remove nan values from a dataframe in python?

2 Answers 2

2 Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related