Python requests_html submit a form by clicking a button using JQuery

Question

I'm exploring Kenneth Reitz's requests_html and trying to submit a form of a JS Rendered Webpage using Jquery. I'm not sure how to do it but, here is my attempt:

from requests_html import HTMLSession
url = "https://example.com"
session = HTMLSession()
r = session.get(url)
r.html.render()
script = """
       () => {
         $("#some_input_field").val("Some value");
         $("#submit_button").click();
         }
         """
r.html.render(script=script, reload=False)

But, the value is not getting set on the input field & it isn't submitting the form... Is there any way to simulate button click or, form submit via xhr in requests_html?

For example: If we use selenium we can simulate button click pretty easily by typing:

element.click()

Wasi · Accepted Answer · 2019-02-26 16:23:19Z

4

Ok, The following code is working in my case:

from requests_html import HTMLSession
url = "https://example.com"
session = HTMLSession()
r = session.get(url)
r.html.render()
script = """
       () => {
              if ( jQuery.isReady ) {  
                   $("#some_input_field").val("Some value");
                   $("#submit_button").click();
              }
        }
         """
r.html.render(script=script, reload=False)

EDIT: A better approach should be:

from requests_html import HTMLSession
url = "https://example.com"
session = HTMLSession()
r = session.get(url)
r.html.render()
script = """
       () => {
              $(document).ready(function() {  
                   $("#some_input_field").val("Some value");
                   $("#submit_button").click();
              })
        }
         """
r.html.render(script=script, reload=False)

edited Feb 26, 2019 at 16:23

answered Feb 26, 2019 at 16:16

Wasi

1,5023 gold badges18 silver badges33 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

Jia Gao Over a year ago

Hey very much appreciate seeing this, could you please explain why and how the part after script= works?

Wasi Over a year ago

@JasonGoal script variable is a string containing JavaScript code. It fills an input field and clicks the submit button. ()=>{} is an arrow function. The dollar sign in the second function is JQuery

Jake Over a year ago

@Wasi Can you explain what document refers to in the second code snippit? I get the following error when I try to run (and have replaced the some_input_field and submit_button...: pyppeteer.errors.ElementHandleError: Evaluation failed: ReferenceError: $ is not defined...

Jake Over a year ago

My script is as follows: ``` script = """ () => { $(document).ready(function() { $("#publish-to-mobile-button").click(); $("#publish-to-mobile-confirm-button").click(); }) } """ ```

Jake Over a year ago

UPDATE: I figured it out with the help of github.com/psf/requests-html/issues/355#issuecomment-942914999

|

Jake · Accepted Answer · 2022-08-17 16:14:39Z

0

The below statements were not made by me but were helpful in resolving. Credit to alairjunior's response on this issue

I wasn't able to use the 'load' and 'DOMContentLoaded' events. Did not investigate why yet. I suspect the script is run using the "console", so it cannot get those events. But this is just a wild guess.

For a quick and dirty solution, I was able to use setTimeout:

script = """
() => {
setTimeout(function(){
    document.querySelectorAll("a")[2].click();
}, 3000);
}
"""

If I use r.html.render(sleep=10,script=script) I am able to get the content of the page after the click was executed.

Hope this is useful.

answered Aug 17, 2022 at 16:14

Jake

2964 silver badges10 bronze badges

Collectives™ on Stack Overflow

Python requests_html submit a form by clicking a button using JQuery

2 Answers 2

6 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

6 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related