How to download a text file or some objects from webpage using Python?

Question

I am writing a function that downloads and stores the today's list of pre-release domains .txt file from http://www.namejet.com/pages/downloads.aspx. I am trying to achieve it using json.

import json
import requests

def hello():
    r = requests.get('http://www.namejet.com/pages/downloads.aspx') 
    #Replace with your website URL

    with open("a.txt", "w") as f: 
    #Replace with your file name
        for item in r.json or []:
            try:
                f.write(item['name']['name'] + "\n") 
            except KeyError: 
                pass  

hello()

I need to download the file which consist of pre-release domains using python. How can I do that? Is the above code right way to do it?

This sounds like a problem with DNS resolution. I am assuming that the indentation is correct in your code, since its not right in your paste. Also, the link you are trying to open doesn't exist (404 error). — Burhan Khalid
– Burhan Khalid, Commented Oct 8, 2012 at 4:20
Your error still has to do with DNS resolution. Once you get that resolved, r.json will be empty since the page doesn't return any json. — Burhan Khalid
– Burhan Khalid, Commented Oct 8, 2012 at 4:28

root · Accepted Answer · 2012-10-08 05:56:25Z

2

I dont't think mechanize is much use for javascript, use selenium. Here's an example:

In [1]: from selenium import webdriver
In [2]: browser=webdriver.Chrome() # Select browser that you want to automate 
In [3]: browser.get('http://www.namejet.com/pages/downloads.aspx')
In [4]: element=browser.find_element_by_xpath(
            '//a[@id="ctl00_ContentPlaceHolder1_hlPreRelease1"]')

In [5]: element.click()

Now you can find prerelease_10-08-2012.txt in your download folder and you can open it in a usual way.

edited Oct 8, 2012 at 5:56

answered Oct 8, 2012 at 5:03

root

81.1k25 gold badges111 silver badges120 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Burhan Khalid · Accepted Answer · 2012-10-08 04:45:53Z

0

I see a few problems with your approach:

The page doesn't return any json; so even if you were to access the page successfully, r.json will be empty:
```
>>> import requests
>>> r = requests.get('http://www.namejet.com/pages/downloads.aspx')
>>> r.json
```
The file that you are after, is hidden behind a postback link; which you cannot "execute" using requests as it will not understand javascript.

In light of the above, the better approach is to use mechanize or alternatives to emulate a browser. You could also ask the company to provide you with a direct link.

answered Oct 8, 2012 at 4:45

Burhan Khalid

175k20 gold badges254 silver badges291 bronze badges

1 Comment

New Folder Over a year ago

Thank you for your answer but it won't download or get my any files. I want to download files under "Pre-Release Domains". Company won't provide me the direct link! So as today is 8th of October you want to get the file "Monday, October 08, 2012".

Collectives™ on Stack Overflow

How to download a text file or some objects from webpage using Python?

2 Answers 2

Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related