Consider I have 1000 .CSV files with names of my employees. So there isn't any order or numbers in file names. Is there a way to say to the computer in Python language, that read files from first to end in a special folder, whit no matter what their name is?
(It's not important for me the data is for who, I only need to grab these data for analyzing).
-
Answer: Yes, it's possible to read all *.csv files in a given folder with Python.Mike Scotty– Mike Scotty2019-02-14 09:19:21 +00:00Commented Feb 14, 2019 at 9:19
-
Actually a duplicate questionNihal– Nihal2019-02-14 09:21:47 +00:00Commented Feb 14, 2019 at 9:21
-
check this answer stackoverflow.com/a/9251091/7053679Nihal– Nihal2019-02-14 09:22:28 +00:00Commented Feb 14, 2019 at 9:22
Add a comment
|
3 Answers
You can read all csv files in a directory like this:
My csv:
col1,col2,col3
a,b,c
d,e,f
Code:
import glob
import csv
PATH = "/Users/stack/"
for file in glob.glob(PATH+"*.csv"):
with open(file) as csvfile:
spamreader = csv.reader(csvfile, delimiter=',')
for row in spamreader:
print(" ".join(row))
Output:
col1 col2 col3
a b c
d e f
Process finished with exit code 0
13 Comments
lovecode
Thanks madik, I tried your code but at this line
PATH = "C:\Users\m\Desktop\TSE" I get SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 2-3: truncated \UXXXXXXXX escape
madik_atma
Check pls this question: stackoverflow.com/questions/2953834/windows-path-in-python
lovecode
The error solved by this addaption
PATH = r"C:\Users\m\Desktop\TSE" and I can build the program with no errors, but there is no special output in other than [Finished in 0.1s]. How can I see the contents of read files?lovecode
May you do what you did with
pandas library too?madik_atma
check pls the updated answer @user3486308
|
use code like:(replace current path (.) with your path:
import os, fnmatch
import csv
listOfFiles = os.listdir('.')
pattern = "*.csv"
for entry in listOfFiles:
if fnmatch.fnmatch(entry, pattern):
with open(entry, newline='') as csvfile:
spamreader = csv.reader(csvfile)
for line in spamreader:
print(line)
import os, fnmatch
import pandas as pd
listOfFiles = os.listdir('.')
pattern = "*.csv"
for entry in listOfFiles:
if fnmatch.fnmatch(entry, pattern):
read_File_as_DF=pd.read_csv(entry)
print(read_File_as_DF)
7 Comments
lovecode
It seems your code read
.txt files not .csv files.Pradeep Pandey
change the .txt it will work for csv as well
lovecode
Great, it worked! Can you write also a pandas version? I think it's simpler to understand and work for beginners on Python!
lovecode
One line of outputs that I get with your code is like this
['13971115', '1020002.00', '1020002', '1020002', '1020002.00', '1020002.00', '1021098', '130', '1'] . How can I delete ` ' ' ` quotes? Also save this read data as a matrix? (I have numpy for matrix working).Pradeep Pandey
use pandas version it will be resolved, just use separator rightly and qua toes if have in read_csv function
|
Yes you can. I would use a simple regex based tester to check the file and so essentially what you're doing is you are using a for loop to go through the directory and using the if-statement, we test the file to see if it contains '.csv'. After this we open the file and we simply append it to our output which you can either choose to analyse or store as a file. I've commented out the option of outputting to a file, however if you wish to you could.
import re
# Redefine this to the path of your folder:
folderPath = "SET UNIX PATH HERE"
output = None
for file in os.listdir(folderPath):
if re.search(r'.csv', file):
with open(file, r) as readFile:
output += readFile.read()
# Uncomment this part if you would like to store the output to a file
# Define the path to the file that will be created:
# outputFilePath = "SET UNIX PATH"
# with open(outputFilePath, w+) as outputFile:
# outputFile.write(output)
Hope this helps :)
