cannot write file with full path in Python

Question

I am using Pandas on Mac, to read and write a CSV file, and the weird thing is when using full path, it has error and when using just a file name, it works. I post my code which works and which not works in my comments below, and also detailed error messages. Anyone have any good ideas?

sourceDf = pd.read_csv(path_to_csv)
sourceDf['nameFull'] = sourceDf['nameFirst'] + ' ' + sourceDf['nameLast']
sourceDf.to_csv('newMaster.csv') # working
sourceDf.to_csv('~/Downloads/newMaster.csv') # not working

Traceback (most recent call last):
  File "/Users/foo/PycharmProjects/DataWranglingTest/CSVTest1.py", line 36, in <module>
    add_full_name(path_to_csv, path_to_new_csv)
  File "/Users/foo/PycharmProjects/DataWranglingTest/CSVTest1.py", line 28, in add_full_name
    sourceDf.to_csv('~/Downloads/newMaster.csv')
  File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/frame.py", line 1189, in to_csv
    formatter.save()
  File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/format.py", line 1442, in save
    encoding=self.encoding)
  File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/common.py", line 2831, in _get_handle
    f = open(path, mode)
IOError: [Errno 2] No such file or directory: '~/Downloads/newMaster.csv'

Tried to use prefix r, but not working,

    path_to_csv = r'~/Downloads/Master.csv'
    path_to_new_csv = r'~/Downloads/Master_new.csv'

  File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/frame.py", line 1189, in to_csv
    formatter.save()
  File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/format.py", line 1442, in save
    encoding=self.encoding)
  File "/usr/local/Cellar/python/2.7.8/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pandas/core/common.py", line 2831, in _get_handle
    f = open(path, mode)
IOError: [Errno 2] No such file or directory: '~/Downloads/Master_new.csv'

thanks in advance, Lin

Michael Blakeley · Accepted Answer · 2015-08-15 23:42:26Z

4

Try using os.path.join().

import os
(...)
output_filename = 'newMaster.csv'
output_path = os.path.join('Downloads', output_filename)
(...)
sourceDf.to_csv(output_path)

Use the same methodology to point pandas.read_csv() in the right direction.

answered Aug 15, 2015 at 23:42

Michael Blakeley

3461 silver badge7 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Lin Ma Over a year ago

tried, path_to_csv = os.path.join("Downloads", "Master.csv") path_to_new_csv = os.path.join("Downloads", "Master_new.csv"), and met with error message, self._engine = CParserWrapper(self.f, **self.options) File "pandas/parser.pyx", line 350, in pandas.parser.TextReader.__cinit__ (pandas/parser.c:3173) File "pandas/parser.pyx", line 594, in pandas.parser.TextReader._setup_parser_source (pandas/parser.c:5912) IOError: File Downloads/Master.csv does not exist

Michael Blakeley Over a year ago

I'm not familiar with OSX pathing conventions. Try os.path.join('~', 'Downloads', etc) ?

Lin Ma Over a year ago

Does not work. :( It seems full path name works (means without abbreviation like '~').

Berserker · Accepted Answer · 2015-08-15 23:53:25Z

1

You didn't specify python version. On 3.4 you can use pathlib, otherwise use os.path.join() or quoting:

sourceDf.to_csv(r'~/Downloads/newMaster.csv')

Notice the r. The problem is that /n is newline, which is not allowed in a path.

answered Aug 15, 2015 at 23:53

Berserker

1,11212 silver badges28 bronze badges

4 Comments

Lin Ma Over a year ago

tried but not working, and I have posted my error message in my original post, and appreciate if you could help to take a look. Thanks.

Berserker Over a year ago

give os.path.join a go, if that also doesn't work, make sure the file actually exists and is reachable for python (read-access rights) It would be os.path.join("~", "Downloads", "newMaster.csv")

Lin Ma Over a year ago

It is very weird I can use '~/Downloads/Master.csv' to read, but cannot use '~/Downloads/Master_new.csv' to write, do you think it is a permission issue?

Berserker Over a year ago

That, or you are not actually opening the output file in write mode.

Collectives™ on Stack Overflow

cannot write file with full path in Python

2 Answers 2

3 Comments

4 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

4 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related