use python requests to submit a form and get results

Question

I am not familiar with requests of python. I need to do one thing. here's a link:https://bugzilla.mozilla.org/buglist.cgi?bug_status=ASSIGNED&bug_status=REOPENED&bug_status=RESOLVED&bug_status=VERIFIED&bug_status=CLOSED&query_format=advanced&resolution=---&resolution=FIXED&resolution=INVALID&resolution=WONTFIX&resolution=DUPLICATE&resolution=WORKSFORME&resolution=INCOMPLETE&resolution=SUPPORT&resolution=EXPIRED&resolution=MOVED&order=bug_id&limit=10

There is a button of xml below. If you click this button, you will get a xml page. I hope I can use python code to automatically get the xml content.

the html on that page of the xml button is :

<form method="post" action="show_bug.cgi">
<input type="hidden" name="ctype" value="xml">
<input type="hidden" name="id" value="35">
<input type="hidden" name="id" value="36">
<input type="hidden" name="id" value="37">
<input type="hidden" name="id" value="38">
<input type="hidden" name="id" value="39">
<input type="hidden" name="id" value="41">
<input type="hidden" name="id" value="42">
<input type="hidden" name="id" value="43">
<input type="hidden" name="id" value="51">
<input type="hidden" name="id" value="61">
<input type="hidden" name="excludefield" value="attachmentdata">
<input type="submit" value="XML" id="xml">
</form>

I tried to use requests. I tried code like this:

import requests
values = { 'submit': 'xml'}
req = requests.post('https://bugzilla.mozilla.org/buglist.cgi?bug_status=ASSIGNED&bug_status=REOPENED&bug_status=RESOLVED&bug_status=VERIFIED&bug_status=CLOSED&query_format=advanced&resolution=---&resolution=FIXED&resolution=INVALID&resolution=WONTFIX&resolution=DUPLICATE&resolution=WORKSFORME&resolution=INCOMPLETE&resolution=SUPPORT&resolution=EXPIRED&resolution=MOVED&order=bug_id&limit=10',data=values)
print req.text

but I got errors.

can someone help me to point out the right usage of requests? Thanks.

Thanks Jud's help. He solved my problem!

user1167910
– user1167910

2013-11-18 17:13:54 +00:00
Commented Nov 18, 2013 at 17:13 — user1167910
– user1167910, Commented Nov 18, 2013 at 17:13

Jud · Accepted Answer · 2013-11-18 17:19:34Z

2

How did you install requests? If you get import errors you probably don't have all the dependencies installed. See the discussion here: https://github.com/kennethreitz/requests/issues/513.

Also, it looks like you're hitting the wrong URL. You don't want to GET the original buglist.cgi, you want to POST to show_bug.cgi, since that's the target of the form action. You also need to include all the values of the hidden form fields:

import requests
values = {}
values['ctype'] = 'xml'
# Multiple values for the same name are handled via array
values['id'] = [35, 36, 37, 38, 39, 41, 42, 43, 51, 61]
values['excludefield'] = 'attachmentdata'
req = requests.post('https://bugzilla.mozilla.org/show_bug.cgi', data=values)
print req.text

edited Nov 18, 2013 at 17:19

answered Nov 18, 2013 at 16:32

Jud

1,2181 gold badge9 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

user1167910 Over a year ago

Thanks for your help. I already fixed the import error. but I don't know how to simulate this action, on the page of bugzilla.mozilla.org/… and click the xml button?

Jud Over a year ago

Did you fix the URL as I also suggested?

user1167910 Over a year ago

Yes @Jud I tried but the page bugzilla.mozilla.org/show_bug.cgi is totally different page(not xml).

Jud Over a year ago

I forgot to add the additional form fields, that's likely the problem.

user1167910 Over a year ago

I guess so. but how to deal with hidden fields? I am so sorry

|

rdodev · Accepted Answer · 2013-11-18 16:47:57Z

0

import requests
values = { 'submit': 'xml'}
req = requests.post('https://bugzilla.mozilla.org/buglist.cgi?bug_status=ASSIGNED&bug_status=REOPENED&bug_status=RESOLVED&bug_status=VERIFIED&bug_status=CLOSED&query_format=advanced&resolution=---&resolution=FIXED&resolution=INVALID&resolution=WONTFIX&resolution=DUPLICATE&resolution=WORKSFORME&resolution=INCOMPLETE&resolution=SUPPORT&resolution=EXPIRED&resolution=MOVED&order=bug_id&limit=10',data=values)
print req.text

seems to work for me.

edited Nov 18, 2013 at 16:47

answered Nov 18, 2013 at 16:20

rdodev

3,2123 gold badges28 silver badges34 bronze badges

5 Comments

user1167910 Over a year ago

I am sorry that I didn't express clearly before. I edited my question. But get still doesn't work.

rdodev Over a year ago

@user1167910 what is the error you are getting? Without the error, I'm shooting in the dark.

user1167910 Over a year ago

sorry, i got an error ImportError: No module named certifi what does it mean?

user1167910 Over a year ago

Oh I fixed that. But I still get some html thing. Not the xml I want.

rdodev Over a year ago

That API call looks very nasty.Check out what the curl command looks like as generated by Chrome. pastebin.com/QBqMsDCH

Collectives™ on Stack Overflow

use python requests to submit a form and get results

2 Answers 2

6 Comments

5 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

5 Comments

Your Answer

Sign up or log in

Post as a guest

Related