Python: URL data to csv reader

Question

I'm trying to convert the results of a url request to a CSV reader object in Python. The output seems fine but the following error occurs:

    for study_id, sample_id, run_id in reader:
ValueError: not enough values to unpack (expected 3, got 1)

First I request the data and convert it to string data:

req = urllib.request.Request(url=url_get_project_runs, headers={'Content-Type': 'text/plain'})
        res = urllib.request.urlopen(req)
        dec_res =  res.read().decode()
        sys.stderr.write(str(dec_res)) --> for the print see below
        return dec_res

The print will give this:

ERP001736,ERS494374,ERR598958
ERP001736,ERS494394,ERR598963
ERP001736,ERS494431,ERR598964
ERP001736,ERS494445,ERR599170
ERP001736,ERS494488,ERR598996
ERP001736,ERS494518,ERR598976
ERP001736,ERS494559,ERR598986
ERP001736,ERS494579,ERR599078
ERP001736,ERS494616,ERR598944
ERP001736,ERS494628,ERR599001
ERP001736,ERS488919,ERR1701760

This seems fine to me because the items are seperated by commas and the "rows"are intact. However if I use this as input for the CSV reader and try to print the three columns, as here:

reader = csv.reader(dec_res, delimiter=',')
        for study_id, sample_id, run_id in reader:
            print(study_id + ", " + sample_id + ", " + run_id)

The following error will occur: ValueError: not enough values to unpack (expected 3, got 1)

I further tested some code to find the problem:

 for row in reader:
            sys.stderr.write(str(row))

This will give:

['E']['R']['P']['0']['0']['1']['7']['3']['6']['', '']['E']['R']['S']['4']['7']...etc

damisan · Accepted Answer · 2017-08-25 10:44:32Z

3

You are passing str to the csv reader and it expects a line-by-line iterable. str is a character-by-character iterable. Wrap your string with StringIO or use splitlines (or any other means to provide line-by-line iterable).

import csv
from io import StringIO

dec_res = """ERP001736,ERS494374,ERR598958
ERP001736,ERS494394,ERR598963
ERP001736,ERS494431,ERR598964
ERP001736,ERS494445,ERR599170
ERP001736,ERS494488,ERR598996
ERP001736,ERS494518,ERR598976
ERP001736,ERS494559,ERR598986
ERP001736,ERS494579,ERR599078
ERP001736,ERS494616,ERR598944
ERP001736,ERS494628,ERR599001
ERP001736,ERS488919,ERR1701760"""

reader = csv.reader(StringIO(dec_res), delimiter=',')
#reader = csv.reader(dec_res.splitlines(), delimiter=',')

for study_id, sample_id, run_id in reader:
    print(study_id + ", " + sample_id + ", " + run_id)

edited Aug 25, 2017 at 10:44

answered Aug 25, 2017 at 10:38

damisan

1,0476 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

CodeNoob Over a year ago

Thankyou so much!

Violet Red Over a year ago

Actually csv can (not always) read normal strings if you pass them inside a list: ` >>>data = csv.reader(['ERP001736,ERS494374,ERR598958']) >>> list(data) [['ERP001736', 'ERS494374', 'ERR598958']] ` But @damians way is much better.

Collectives™ on Stack Overflow

Python: URL data to csv reader

1 Answer 1

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related