Unsuccessful post to asp.net form using Python requests

Question

I am attempting to use Python requests to post a value to the "employer" form in the http://www.myvisajobs.com/Search_Visa_Sponsor.aspx?N=

This is what I have tried so far in Python:

import requests
url = "http://www.myvisajobs.com/Search_Visa_Sponsor.aspx?N="
data = {"ctl00$ctl00$ContentPlaceHolder1$ContentPlaceHolder1$txtCompany":"Microsoft"}
r = requests.post(url,data)
print(r.text)

Which returns only the original HTML. I am trying to return the resulting HTML. My gut feeling is I am doing something fundamentally wrong, but I am not sure what.

I've edited the tags to reflect that. Thanks.

user32882
– user32882

2016-01-13 04:54:23 +00:00
Commented Jan 13, 2016 at 4:54 — user32882
– user32882, Commented Jan 13, 2016 at 4:54

alecxe · Accepted Answer · 2016-01-13 05:14:28Z

1

There are much more parameters sent in the search POST request than just the ctl00$ctl00$ContentPlaceHolder1$ContentPlaceHolder1$txtCompany referring to the company name.

Instead, to make things transparent and easy, I would use RoboBrowser that would "auto-fill" other form POST parameters needed. Example working code:

from robobrowser import RoboBrowser


url = "http://www.myvisajobs.com/Search_Visa_Sponsor.aspx?N="

browser = RoboBrowser(history=True)
browser.open(url)

form = browser.get_form(id='aspnetForm')
form['ctl00$ctl00$ContentPlaceHolder1$ContentPlaceHolder1$txtCompany'].value = 'Microsoft'
browser.submit_form(form)

results = browser.select('div#ctl00_ctl00_ContentPlaceHolder1_ContentPlaceHolder1_divContent table tr')[1:]
for result in results:
    cells = result.find_all("td")

    print(cells[2].get_text(strip=True))

It prints the company names from the search results:

Microsoft Corporation
Microsoft Operations Puerto Rico, Llc
Microsoft Caribbean, Inc.
Standard Microsystems Corporation
4Microsoft Corporation
Microsoft Business Solutions Corporation
Microsoft C98052orporation
Microsoft Ccrporation
Microsoft Coiporation
Microsoft Copporation
Microsoft Corforation
Microsoft Licensing, GP
Microsoft Way
Microsoftech Inc
Quantitative Micro Software Llc
Webtv Networks Microsoft Sub
Microsoft
FAST, A Microsoft Subsidiary
Microsoft Corporation - Sham
Microsoft Partner Careers (sponsored By Microsoft Dynamics)
Microsoft Iberica
Microsoft Karthi

answered Jan 13, 2016 at 5:14

alecxe

476k127 gold badges1.1k silver badges1.2k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

user32882 Over a year ago

If you don't mind... What are the other parameters getting sent?

alecxe Over a year ago

@user32882 just open browser developer tools, network tab, submit a form and inspect the POST request that would appear in the network grid. Hope that helps.

Collectives™ on Stack Overflow

Unsuccessful post to asp.net form using Python requests

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related