How to append list values to a URL in Python

Question

I have a base url that I would like to append list values to so that I have a list of URLs to scrape. My list comes from a json file, and looks like:

[{u'url': [u'/location/subfile/file1.htm', u'/location/subfile/file2.htm', u'/location/subfile/file3.htm', u'/location/subfile/file4.htm']}]

My base URL is something like http://example.com/placeforfiles/

What I want, ultimately, is a collection of URLs that have that base URL plus the list values, like so:

http://example.com/placeforfiles/location/subfile/file1.htm
http://example.com/placeforfiles/location/subfile/file2.htm
http://example.com/placeforfiles/location/subfile/file3.htm
http://example.com/placeforfiles/location/subfile/file4.htm

There may be thousands of list values I need to append, so I know I need to loop through them and append them but I haven't found a solution that works. I'm currently trying:

import json

with open ('returned_items.json') as links:
    data = json.load(links)

base_url = 'http://example.com/placeforfiles/{}'

for i in data:
    url = 'http://example.com/placeforfiles/{}'.format(i)
    print url

Which is returning:

http://example.com/placeforfiles/({u'url': [u'/location/subfile/file1.htm', u'/location/subfile/file2.htm', u'/location/subfile/file3.htm', u'/location/subfile/file4.htm']},)

Sorry, just realized that was in there by accident I edited the question to remove it. — E. Randy
– E. Randy, Commented Jun 15, 2017 at 17:55

Menachem Hornbacher · Accepted Answer · 2017-06-15 17:58:08Z

1

this is because the dict is the first element in the array. The loop should be for i in data[0]["url"]

answered Jun 15, 2017 at 17:58

Menachem Hornbacher

2,1752 gold badges25 silver badges41 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Sanjay · Accepted Answer · 2017-06-15 18:06:54Z

1

#replcace data with below line
data = json.loads(links)

#replace your last loop with below
if data and 'url' in data[0]:
 for i in data[0]['url']:
  url = 'http://example.com/placeforfiles{}'.format(i)
  print(url)

answered Jun 15, 2017 at 18:06

Sanjay

2,0482 gold badges19 silver badges26 bronze badges

Collectives™ on Stack Overflow

How to append list values to a URL in Python

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related