Processing data from API to CSV

Question

I have this code which gets data from an API and writes to a CSV. However, the data doesnt update when I change the data from the API ( say obtain data for a different car). It seems as though the "struct" variable is hard coded and doesnt change. I want the format to stay the same, but simply update depending on data that is passed through

output data from API :

FoundCategories[]
PageSize1
Page1
List[{'ExteriorColour': None, 'CategoryPath': '/Trade-Me-Motors/Cars/Bentley', 'Subtitle': 'sdf', 'BestContactTime': None, 'StartPrice': 100.0, 'Doors': 0, 'Fuel': None, 'BodyStyle': 'Coupe', 'WofExpires': '/Date(0)/', 'NumberPlate': None, 'ImportHistory': None, 'Transmission': 'Manual', 'EngineSize': 0, 'ListingLength': None, 'StereoDescription': None, 'Category': '0001-0268-7081-', 'Title': 'Bentley Continental 1999', 'Owners': 0, 'IsDealer': False, 'Cylinders': 0, 'AsAt': '/Date(1457728757951)/', 'Odometer': 2000, 'Vin': None, 'Year': 1999, 'StartDate': '/Date(1457326119847)/', 'Region': 'Manawatu', 'Model': 'Continental', 'PriceDisplay': '$100.00', 'Suburb': 'Palmerston North', 'EndDate': '/Date(1457930919847)/', 'RegistrationExpires': '/Date(0)/', 'NoteDate': '/Date(0)/', 'ListingId': 4550689, 'Make': 'Bentley'}]
TotalCount1

Code to write to csv

r3 = requests.get('https://api.website.com', params=myhheadersaders2)

struct = r3.json()

car_data = struct.get('List')
fieldnames=car_data[0].keys()

struct = {'TotalCount': 1, 'PageSize': 1, 'FoundCategories': [], 'Page': 1, 'List': [{'AsAt': '/Date(1457733660023)/', 'Model': 'Continental', 'Suburb': 'Palmerston North', 'NoteDate': '/Date(0)/', 'PriceDisplay': '$100.00', 'EndDate': '/Date(1457930919847)/', 'RegistrationExpires': '/Date(0)/', 'StartPrice': 100.0, 'Owners': 0, 'ListingLength': None, 'CategoryPath': '/Trade-Me-Motors/Cars/Bentley', 'ListingId': 4550689, 'Subtitle': 'sdf', 'Category': '0001-0268-7081-', 'StartDate': '/Date(1457326119847)/', 'Year': 1999, 'WofExpires': '/Date(0)/', 'ExteriorColour': None, 'Vin': None, 'EngineSize': 0, 'Doors': 0, 'BodyStyle': 'Coupe', 'Title': 'Bentley Continental 1999', 'IsDealer': False, 'Make': 'Bentley', 'Transmission': 'Manual', 'Fuel': None, 'ImportHistory': None, 'Odometer': 2000, 'StereoDescription': None, 'Region': 'Manawatu', 'BestContactTime': None, 'Cylinders': 0, 'NumberPlate': None}]}



car_info = struct.get('List')
with open('car_info.csv', 'w') as csvfile:
    writer = csv.DictWriter(csvfile, fieldnames=car_info[0].keys())

    writer.writeheader()
    for row in car_info:
        writer.writerow(row)

Burhan Khalid · Accepted Answer · 2016-03-13 06:22:54Z

You are overwriting the data you receive from the API in your code, because you keep re-using the struct variable.

It looks like you were debugging your csv writing code with a sample data set, and then forgot to remove that part of the code when you added the parts that fetch the data from the API.

I have cleaned up your code and commented out the duplicate lines that were causing the same information to be written to your output:

r3 = requests.get('https://api.website.com', params=myhheadersaders2)
struct = r3.json()  # this is the data from the API

car_data = struct.get('List')
fieldnames=car_data[0].keys()

# now you are simply overwriting it with some sample data,
# ignoring whatever is returned from the API
# struct = {'TotalCount': 1, ...}]}

# this is the same as car_data = struct.get('List')
# car_info = struct.get('List')

with open('car_info.csv', 'w') as csvfile:
    writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
    writer.writeheader()
    writer.writerows(car_data)

Collectives™ on Stack Overflow

Processing data from API to CSV

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related