Problems Using VBA to Submit a Web Page - Using the click button function but web page won't submit

Question

I am writing a VBA code to pull data from a website (https://app.buzzsumo.com/top-content). I have a functional code that runs without errors however I still can't get the webpage to actually submit the form when the click command runs. I have tried many different approaches and combinations of submitting the form/clicking the submit button but none have seemed to work so far. Below is my current code.

 Sub clickFormButton()
 Dim ie As Object
 Dim form As Variant, 
 Dim button As Variant

'add the “Microsoft Internet Controls” reference in VBA Project
 Set ie = CreateObject("InternetExplorer.Application")

'using input box to enter URL I am serching for
Search_URL = InputBox("Enter URL to Search For")

With ie
.Visible = True
.navigate ("https://app.buzzsumo.com/#/top-content")

'Ensure that the web page downloads completely 
 While ie.ReadyState <> 4
 DoEvents
 Wend

'assigning the input variables to the html elements of the form
 ie.document.getElementsByName("q").Item.innertext = Search_URL

'finding and clicking the button
Set objInputs = ie.document.getElementsByTagName("input")
For Each ele In objInputs
   If ele.Title Like "Press Enter to Search" Then
        ele.Click
    End If

End With
End Sub

I have also tried other methods to find and click the button such as:

'Dim i As Variant
'Set form = ie.document.getElementsByClassName("btn btn-highlight")

'For i = 1 To 5
'If form.Item(i).DefaultValue = "Search!" Then
     'Set button = form.Item(i)
     'button.Click
'End If
'Next i

Please provide any recomendations on what I may be missing or how I can get this code to actually submit the form and advance to the search results. Thanks in advance for any help you can provide!

Here are some additional details: Unfortunately the element I am trying to click (the "Search" button) does not have an ID or Name associated with it. This is why is was trying alternative approaches, such as looping through all of the object and trying to find the one with the right “Title”. Here is the code for the element from the DOM explorer:

<input title="Press Enter to search" class="btn btn-highlight" type="submit" ng-disabled="topContentSearchForm.$invalid" value="Search!"/>

The only attributes associated with it are:

class: btn btn-highlight
type: submit
ng-disabled: topContentSearchForm.$invalid
value: Search!
title: Press Enter to Search

Please let me know if there is another way to find the element ID/name? or if there is another way to click the button without these attributes? Thanks

You need to identify the appropriate HTML element. What is the element and what are its properties in the HTML? — David Zemens
– David Zemens, Commented Feb 5, 2015 at 2:12
Thank you David, I have added the HTML element information to the post above, along with a more detailed explanation regarding what is happening. Do you have any more suggestions as to how I can successfully click this Search button without the element ID or name? Thanks! — Michael Droesch
– Michael Droesch, Commented Feb 6, 2015 at 14:31
This may require form.submit but I don't have time to look at this at the moment. I will try to check it out later this afternoon. — David Zemens
– David Zemens, Commented Feb 6, 2015 at 14:51
Ok Thanks David! I really apprecaite any input you can provide. — Michael Droesch
– Michael Droesch, Commented Feb 6, 2015 at 15:55

Kdunc2015 · Accepted Answer · 2015-07-14 14:34:35Z

3

I know this is an old post but... I have been using this effectively..

   'click login
            Set htmlDoc = .document
            Set htmlColl = htmlDoc.getElementsByTagName("input")
            Do While htmlDoc.readyState <> "complete": DoEvents: Loop
                For Each htmlInput In htmlColl
                    If Trim(htmlInput.Type) = "submit" Then
                        htmlInput.Click
                        Exit For
                    End If
                Next htmlInput

answered Jul 14, 2015 at 14:34

Kdunc2015

1931 gold badge2 silver badges9 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Jeanno · Accepted Answer · 2015-02-05 15:15:54Z

2

A couple of ideas:

   While ie.ReadyState <> 4
       DoEvents
   Wend

If you have javascripts on the page use Application.Wait Now + TimeSerial(0, 0, 4) (basically wait for 4 seconds) instead.

Second I don't understand why you need to loop through all the objects on the web page. The easier way would be to go that webpage in IE, hit F12 and select element in DOM explorer, you can get the ID or Name of the button and then use ie.document.GetElementByID("buttonID").Click or ie.document.GetElementsByName("buttonName").Item.Click

Let me know if this helps.

Edit: After inspecting the particular webpage it appears that the ID and Name attributes for that button are missing. So I had to resort to the following:

Dim i As integer
Set form = ie.document.getElementsByClassName("btn btn-highlight")
On Error Resume Next
For i = 1 To 20
If form.Item(i).DefaultValue = "Search!" Then
     form.Item(i).Click
End If
Next i

The relevant button is clicked for the fourth item (I had to manually go through the loop because 3rd item navigated away from the page to a pricing page, so i had to go back). Anyway the full code is the following, please note that you will need to go through this exercise again if there were changes to the webpage

Sub clickFormButton()
    Dim ie As Object
    Dim form As Variant
    Dim button As Variant

    'add the “Microsoft Internet Controls” reference in VBA Project
    Set ie = CreateObject("InternetExplorer.Application")
    'using input box to enter URL I am serching for
    Search_URL = InputBox("Enter URL to Search For")

    With ie
        .Visible = True
        .navigate ("https://app.buzzsumo.com/#/top-content")
    End With
    'wait for page to load
    Application.Wait Now + TimeSerial(0, 0, 5)
    'assigning the input variables to the html elements of the form
    ie.document.getElementsByName("q").Item.InnerText = Search_URL
    'finding and clicking the button
    ie.document.getElementsByClassName("btn btn-highlight").Item(4).Click
End Sub

edited Feb 5, 2015 at 15:15

answered Feb 5, 2015 at 2:11

Jeanno

2,8894 gold badges26 silver badges33 bronze badges

4 Comments

Michael Droesch Over a year ago

Hi Jeanno, Thanks for the suggestions! as far as the element ID or Name is concerned, the DOM explorer does not list these attributes for the element. I have added additional details above, including the code from the DOM explorer. Please let me know if you have any other suggestions, Thanks again!

Jeanno Over a year ago

Please check my updated reply and mark it as answer if you are satisfied with it.

Michael Droesch Over a year ago

Hi Jeanno, Thanks for this updated code, I have been able to run it with no errors, however when the code completes it still does not navigate me to the search results page, does it work when you run it? perhaps I am missing something? I also manually ran the looping code you provided for i = 1 to 20, and I had the same thing happen, it navigated to a pricing page on item number 3, so we are on the same page with that part. I'm just not sure why it does not navigate to the search results page when I use your full code. Thanks again, any other recommendations would be greatly appreciated.

David Zemens Over a year ago

Application.Wait is usually not ideal. A ready-waiting loop, like Do While Not Ie.ReadyState=4 And Not IE.busy: Loop is generally preferable :)

David Zemens · Accepted Answer · 2015-02-06 18:37:04Z

It looks like you could potentially just build the string URL, for example if you put "abcd" in the search field, the resulting URL will be:

https://app.buzzsumo.com/top-content?result_type=total&type=articles&num_days=360&tfc=false&general_article&infographic&video&page=1&guest_post&giveaway&interview&links_sitewide=true&unique_domains=true&backlinks=false&q=abcd&offset=0

Note the bolded portion which is the search query.

So, and this is just a quick idea that may work as long as you're not trying to abuse their system by sending 1000's of automated requests:

Sub FetchWebsite()
    Dim ie As Object
    Dim form As Variant
    Dim button As Variant
    Dim url As String

    'add the “Microsoft Internet Controls” reference in VBA Project
    Set ie = CreateObject("InternetExplorer.Application")
    'using input box to enter URL I am serching for
    Search_URL = InputBox("Enter URL to Search For")

    '### BUILD THE FULL URL
    url = "https://app.buzzsumo.com/top-content?result_type=total&type=articles&num_days=360&tfc=false&general_article&infographic&video&page=1&guest_post&giveaway&interview&links_sitewide=true&unique_domains=true&backlinks=false&q=" & Search_URL & "&offset=0"

    With ie
        .Visible = True
        .navigate url
    End With
    'wait for page to load
    Do
    Loop While Not ie.ReadyState = 4 And Not ie.Busy

    AppActivate "Internet Explorer"

End Sub

I did some poking around in the Locals window and this should also work, modified from your code. This would be the Form.Submit that I mentioned in comment on OP.

Sub clickFormButton()
    Dim ie As InternetExplorer
    Dim form As Variant
    Dim button As Variant

    Dim ele As HTMLFormElement
    'add the “Microsoft Internet Controls” reference in VBA Project
    Set ie = CreateObject("InternetExplorer.Application")
    'using input box to enter URL I am serching for
    Search_URL = InputBox("Enter URL to Search For")

    With ie
        .Visible = True
        .navigate ("https://app.buzzsumo.com/#/top-content")
    End With
    'wait for page to load
    Do
    Loop While Not ie.ReadyState = 4 And Not ie.Busy

    'assigning the input variables to the html elements of the form
    ie.document.getElementsByName("q").Item.InnerText = Search_URL
    'finding and clicking the button

    ie.document.getElementsByClassName("btn btn-highlight").Item(4).form.submit
End Sub

Thank you David, This worked and solved my problem perfectly!!... and no im only searching for about 20 websites once a week which I could certianly do mannually but this is much quicker. Thank you so much!
This also works, from your original code: ie.document.getElementsByClassName("btn btn-highlight").Item(4).form.submit

QHarr · Accepted Answer · 2018-06-29 19:06:03Z

1

CSS selector:

You can use CSS selector of #search-btn > div. Which is div within className search-btn. "#" means class.

VBA:

Use .querySelector method to apply CSS selector:

ie.document.querySelector("#search-btn > div").Click

answered Jun 29, 2018 at 19:06

QHarr

84.5k14 gold badges58 silver badges105 bronze badges

Collectives™ on Stack Overflow

Problems Using VBA to Submit a Web Page - Using the click button function but web page won't submit

4 Answers 4

Comments

4 Comments

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Comments

4 Comments

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related