I have a create a python script to clean a csv file. code in the script file "CleanCSV.py" as below
import csv
filepath_i = 'C:\Source Files\Data Source\Flat File Source\PatientRecords.csv'
filepath_o = 'C:\Python\PatientRecords.csv'
rows = []
with open(filepath_i, 'r', newline='') as csv_file:
csv_reader = csv.reader(csv_file, delimiter=',', quotechar='"')
with open(filepath_o, 'w', newline='' ) as writeFile:
writer = csv.writer(writeFile, lineterminator='\r')
for row in csv_reader:
row[3] = row[3].replace("\n","").replace("\r","")
rows.append(row)
writer.writerows(rows)
This is working fine when ran from python editor. but not creating file when ran from command line like below.
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64>python C:\Python\CleanCSV.py
I tried this as well
C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64>pythonw C:\Python\CleanCSV.pyw
I provided full access to folder but still its not creating any file at destination. Please let me know if I am missing anything.
Also please suggest if this code can be optimized. I can't use external packages like pandas so I did it with csv. Thanks in advance.
Extension
When I changed to x for write setting
with open(filepath_o, 'x', newline='' ) as writeFile:
to my surprise I got this error
File "CleanCSV.py", line 8, in <module>
with open(filepath_o, 'x', newline='' ) as writeFile:
FileExistsError: [Errno 17] File exists: 'C:\\Python\\PatientRecords1.csv'
but I don't see the file in the directory. even after setting hidden files to true. So I ran this script.
from pathlib import Path
config = Path(filepath_o )
if config.is_file():
print('yes')
print(config)
else:
print('no')
got this ouptut, but there is no file in the directory!! puzzled.
yes
C:\Python\PatientRecords1.csv
Extension 2
Rewrote script to try with directories
with open(filepath_i,'r') as csv_file:
csv_reader = csv.DictReader(csv_file, delimiter=',', quotechar='"')
with open('PatientRecords1.csv', 'w') as writeFile:
fieldnames = ['DRG Definition','Provider Id','Provider Name','Provider Street Address','Provider City','Provider State','Provider Zip Code','Hospital Referral Region Description','Hospital Category','Hospital Type', 'Total Discharges' ,'Covered Charges' , 'Total Payments' ,'Medicare Payments']
writer = csv.DictWriter(writeFile,fieldnames=fieldnames)
for row in csv_reader:
row['Provider Street Address'] = row['Provider Street Address'].replace("\n","").replace("\r","")
writer.writerows(row)
But received this error
Traceback (most recent call last):
File "CleanCSV.py", line 36, in <module>
writer.writerows(row)
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\csv.py", line 158, in writerows
return self.writer.writerows(map(self._dict_to_list, rowdicts))
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\lib\csv.py", line 148, in _dict_to_list
wrong_fields = rowdict.keys() - self.fieldnames
AttributeError: 'str' object has no attribute 'keys'
Sample input file data
DRG Definition,Provider Id,Provider Name,Provider Street Address,Provider City,Provider State,Provider Zip Code,Hospital Referral Region Description,Hospital Category,Hospital Type, Total Discharges ,Covered Charges , Total Payments ,Medicare Payments
039 - EXTRACRANIAL PROCEDURES W/O CC/MCC,10001,SOUTHEAST ALABAMA MEDICAL CENTER,1108 ROSS CLARK CIRCLE,DOTHAN,AL,36301,AL - Dothan,Specialty Centers,Government Funded,91,"$32,963.07 ","$5,777.24 ","$4,763.73 "
039 - EXTRACRANIAL PROCEDURES W/O CC/MCC,10005,MARSHALL MEDICAL CENTER SOUTH,"2505 U S HIGHWAY
431 NORTH",BOAZ,AL,35957,AL - Birmingham,Specialty Centers,Private Institution,14,"$15,131.85 ","$5,787.57 ","$4,976.71 "
039 - EXTRACRANIAL PROCEDURES W/O CC/MCC,10006,ELIZA COFFEE MEMORIAL HOSPITAL,205 MARENGO STREET,FLORENCE,AL,35631,AL - Birmingham,Rehabilitation Centers,Private Institution,24,"$37,560.37 ","$5,434.95 ","$4,453.79 "
Extension 3
Looks like file is created in the directory, I got output for this two peices of code. However I am unable to see that file, wondering why!!
with open(filepath_o,'r') as csv_file:
csv_reader = csv.reader(csv_file, delimiter=',', quotechar='"')
for row in csv_reader:
print(row)
import os.path
from os import path
print(path.exists(filepath_o ))