Python; Pandas read_csv() not working when file has less than ~200 rows

Question

When I download my CSV file from the FTP server and it contains ~200-500 rows my application works just fine, but when I download a CSV file from the FTP server with only ~1-200 rows I get the error: No columns to parse from file

The CSV contains the headers: "example1";"example2";"example3";"example4"

with rows looking like: "8481832";2022-30-06";GFBG21";4,00

    with open(csv_filename, 'wb') as file:
        ftp.retrbinary(f"RETR {csv_filename}", file.write)
        file_data = pd.read_csv(csv_filename, sep=';', header=0)

Both files are fully identical to each other, just a different amount of rows

Does anyone know if this is a bug, or am i doing something wrong?

Corralien · Accepted Answer · 2022-06-30 10:39:16Z

1

I'm surprised it works for files with line count > 200. You use a context manager and you don't wait the file is closed before reading it. It probably works for other files because the system writes the buffer on disk. You have to de-indent your last line to wait the file be closed properly:

Try:

    with open(csv_filename, 'wb') as file:
        ftp.retrbinary(f"RETR {csv_filename}", file.write)
    file_data = pd.read_csv(csv_filename, sep=';', header=0)

answered Jun 30, 2022 at 10:39

Corralien

121k8 gold badges44 silver badges69 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Jordy V Over a year ago

That worked! Your explanation makes sense why it would work for bigger files but not with the small ones. thank you

Corralien Over a year ago

"this would work for larger files". Don't be sure of that :-). Certainly, you read lines, but I think you didn't read the whole file and missed the last lines for big file.

Collectives™ on Stack Overflow

Python; Pandas read_csv() not working when file has less than ~200 rows

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related