Get CSV file from URL and convert it to array - Python 2.7

Question

I'm trying to get earthquake data, and turn it into an array so I can use that data to visualise earthquakes on a map. I'm writing this script:

import requests
import csv


def csv_to_array(a):
    b = requests.get(a)
    my_file = open(b, "rb")
    for line in my_file:
        el = [i.strip() for i in line.split(',')]
        return el

which I import into another module, and:

import csvToArray
data = csvToArray.csv_to_array(
"http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_hour.csv")
i = 1
while i < len(data):
    stuff = data[i].split(',')
    print stuff[1], stuff[2]
    lat = float(stuff[1])
    lon = float(stuff[2])
    x = webMercX(lon, zoom) - cx
    y = webMercY(lat, zoom) - cy
    i += 1

The other functions of above script are unnecessary but when I run it, I get the following error.

while i < len(data):
TypeError: object of type 'NoneType' has no len()

Because print doesn't returns anything :), You are not returning anything from csv_to_array — ZdaR
– ZdaR, Commented Feb 22, 2017 at 17:10
Wow, that was a stupid mistake (which I fixed just now), but now I get the following error: my_file = open(b, "rb") TypeError: coercing to Unicode: need string or buffer, Response found It's not recognizing the URL as a string — Oxide
– Oxide, Commented Feb 22, 2017 at 17:13
Now you're just returning the first line of the file. The function immediately ends on the first return — Nick T
– Nick T, Commented Feb 22, 2017 at 17:17

Nick T · Accepted Answer · 2017-02-22 17:25:09Z

1

Most suggestions are comments in the code, but a few general ones:

Use better names
Return immediately quits out of the function, if you use yield you can generate line after line

New code with learning experiences:

def csv_to_array(url): # use descriptive variable names
    response = requests.get(url)
    lines = response.text.splitlines() # you don't need an open...the data is already loaded
    for line in lines[1:]: # skip first line (has headers)
        el = [i.strip() for i in line.split(',')]
        yield el # don't return, that immediately ends the function

data = csv_to_array("http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_hour.csv")

for row in data: # don't use indexes, just iterate over the data
    # you already split on commas.
    print(row[1], row[2]) # again, better names
    lat = float(row[1])
    lon = float(row[2])
#     x = webMercX(lon, zoom) - cx
#     y = webMercY(lat, zoom) - cy

Code for the lazy:

import pandas as pd
pd.read_csv('http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_hour.csv')

answered Feb 22, 2017 at 17:25

Nick T

26.9k14 gold badges88 silver badges128 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

gregory Over a year ago

For plotting on google map, I'd take a look at gmplot.

alexey · Accepted Answer · 2017-02-22 17:23:50Z

0

You can replace your first function with generator that iterates over a response data and yields arrays for each line of the file

def csv_to_array(a):
    response = requests.get(a) 
    # you can access response's body via text attribute
    for line in response.text.split('\n'):
        yield [i.strip() for i in line.split(',')]


list(csv_to_array(some_url))

answered Feb 22, 2017 at 17:23

alexey

7165 silver badges9 bronze badges

Collectives™ on Stack Overflow

Get CSV file from URL and convert it to array - Python 2.7

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related