How to scrape JSON data from HTTP response and store it in excel/text file with python

Question

How I can scrape JSON data from following response and save it in excel/text file with python ?

HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
X-Powered-By: Servlet 2.5; JBoss-5.0/JBossWeb-2.1
Content-Disposition: attachment
Content-Type: application/json;charset=utf-8
Content-Length: 797
Date: Sun, 06 Nov 2016 08:32:13 GMT
Via: HTTP/1.1 sophos.http.proxy:3128
Connection: keep-alive

//OK[3,3,2,3,1,["java.util.ArrayList/4159755760","123456789","Bruce Wayne","Male","John","NO","1303052","[email protected]","123456789","A+","NO","No","00","American","63005541113","ssc.client.PersonalDetailValuesManager/227521317","20/05/1996","California","445302","USA"],0,7]

I want to scrape data such as 123456789, Bruce Wayne, Email, etc. in excel/text file, but how I can do it with python? (If not in any programming language)

FYI It’s scrape (and scraping, scraper, scraped) not scrap — DisappointedByUnaccountableMod
– DisappointedByUnaccountableMod, Commented May 8, 2021 at 9:31

Maximilian Peters · Accepted Answer · 2016-11-06 12:09:30Z

2

For Python3 use, urllib.request, json and xlsxwriter

import urllib.request
import json

#read the JSON response
req = urllib.request.urlopen('http://headers.jsontest.com/')
answer = req.read()
answer_json = json.loads(answer.decode())

#save as a text file
with open('json_dump.txt', 'w') as f:
    json.dump(answer_json, f)

#now write to Excel
import xlsxwriter

workbook = xlsxwriter.Workbook('json_dump.xlsx')
worksheet = workbook.add_worksheet()

row = 0
col = 0

for key in answer_json.keys():
    worksheet.write(row, col, key)
    worksheet.write(row, col + 1, answer_json[key])
    row += 1
workbook.close()

answered Nov 6, 2016 at 12:09

Maximilian Peters

31.8k12 gold badges95 silver badges102 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How to scrape JSON data from HTTP response and store it in excel/text file with python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related