CSV Should Return Strings, Not Bytes Error

Question

I am trying to read CSV files from a directory that is not in the same directory as my Python script.

Additionally the CSV files are stored in ZIP folders that have the exact same names (the only difference being one ends with .zip and the other is a .csv).

Currently I am using Python's zipfile and csv libraries to open and get the data from the files, however I am getting the error:

Traceback (most recent call last):   File "write_pricing_data.py", line 13, in <module>
    for row in reader:
_csv.Error: iterator should return strings, not bytes (did you open the file in text mode?)

My code:

import os, csv
from zipfile import *

folder = r'D:/MarketData/forex'
localFiles = os.listdir(folder)

for file in localFiles:
    zipArchive = ZipFile(folder + '/' + file)

    with zipArchive.open(file[:-4] + '.csv') as csvFile:
        reader = csv.reader(csvFile, delimiter=',')

        for row in reader:
            print(row[0])

How can I resolve this error?

have you opened the files in a text editor to make sure the contents are as you expect? — abcd
– abcd, Commented May 1, 2016 at 20:11
Yea, their pretty large, but still standard csv, comma delimiters and all. — ng150716
– ng150716, Commented May 1, 2016 at 20:18
it'd probably be helpful for you to post the full traceback of the error. — abcd
– abcd, Commented May 1, 2016 at 20:19
Open this way instead: with zipArchive.open(file[:-4] + '.csv', 'rt') as csvFile: — Brian Cain
– Brian Cain, Commented May 1, 2016 at 20:24

Jon Clements · Accepted Answer · 2016-05-01 20:33:11Z

1

It's a bit of a kludge and I'm sure there's a better way (that just happens to elude me right now). If you don't have embedded new lines, then you can use:

import zipfile, csv

zf = zipfile.ZipFile('testing.csv.zip')
with zf.open('testing.csv', 'r') as fin:
    # Create a generator of decoded lines for input to csv.reader
    # (the csv module is only really happy with ASCII input anyway...)
    lines = (line.decode('ascii') for line in fin)
    for row in csv.reader(lines):
        print(row)

answered May 1, 2016 at 20:33

Jon Clements

143k34 gold badges254 silver badges288 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

CSV Should Return Strings, Not Bytes Error

1 Answer 1

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related