pandas dataframe to csv export issue

Question

Getting a strange error, In jupyter notes I can run:

df1.to_csv("{0}{1}.csv".format(report_path,db))

and my csv comes out fine.

When I try execute the same thing outside of jupyter by putting the above into a file an external file and running this:

    #imports requred to run querys
    import pandas as pd
    from pandas import DataFrame,Series
    import numpy as np
    from pyhive import presto
    import matplotlib.pyplot as plt
    import seaborn as sn    

 #run config file, which contains the query to generate the report
    def run_config(db):
        print args.an
        print ("config is = {0}".format(config))
        with open(config) as cfg:
            v = cfg.read()
        exec v

I get this error:

lib.write_csv_rows(self.data, ix, self.nlevels, self.cols, self.writer)
File "pandas/_libs/lib.pyx", line 1035, in pandas._libs.lib.write_csv_rows
UnicodeEncodeError: 'ascii' codec can't encode characters in position 8-11: ordinal not in range(128)

I'm fairly certain its something in my DF that is causing this error because other df don't have this issue. But I'm lost as how to fix or edit the code to capture this.

the fix

df1.to_csv("{0}{1}.csv".format(report_path,db), encoding='utf8-8')

Ami Tavory · Accepted Answer · 2018-05-15 22:26:53Z

1

I would try changing the code to

df1.to_csv("{0}{1}.csv".format(report_path,db), encoding='utf8-8')

It seems clear you're encountering characters which aren't ASCII
My guess is that your Jupyter is running a different interpreter than your script
For non-ASCII encoding, UTF8 is a good first bet; if it doesn't work, have a look at Unicode, Dammit.

answered May 15, 2018 at 22:26

Ami Tavory

76.7k13 gold badges152 silver badges196 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

chowpay Over a year ago

thats a good guess and I think the correct one. I didnt even know you can add encoding options to the to_csv function thanks

Collectives™ on Stack Overflow

pandas dataframe to csv export issue

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related