Unable to write pandas Dataframes to csv, getting IO error

Question

I have a pandas Df that I am trying to write to csv but unable to. Here is my code:

_bytes = c_df.to_csv(None).encode()
with fs.open("s3://bucket/key1/_df.csv", 'wb') as f:
        f.write(_bytes)

sectechguy · Accepted Answer · 2018-12-17 21:44:28Z

1

Have you tried using pandas:

c_df.to_csv('s3://bucket/key1/_df.csv', index=False, header=True)

answered Dec 17, 2018 at 20:35

sectechguy

2,1276 gold badges30 silver badges66 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

It says 'str' object has no attribute 'to_csv'

Atihska · Accepted Answer · 2018-12-18 00:17:36Z

1

This helped me:

c_df.to_csv('_df.csv', index=False)  # Preserves the header if present

And then uploading separately to s3 using boto3:

s3 = boto3.resource('s3')
s3.meta.client.upload_file(closed_aoi_filename, s3_bucket, os.path.join(key, closed_aoi_filename))

I tried direct solutions to upload pandas df to s3 but wasn't successful. In case anyone has better answers, please let me know. So far this works.

answered Dec 18, 2018 at 0:17

Atihska

5,21810 gold badges71 silver badges109 bronze badges