Get HTML from website using DOM

Question

I'm trying to write a script in powershell that i can use to get the HTML from a website after a query has been run.

On my own machine i can run the below which works with no issues, but when i try to run it on a Server 2008 machine i get no output from the Document.Body.InnerHTML command, all the Document.GetElementById parts work with no issues.

    $ie = New-Object -com InternetExplorer.Application
    $ie.silent = $true
    $ie.navigate2("http://www.mxtoolbox.com/")
    while($ie.busy) {start-sleep 1}
    $ie.Document.getElementById("ctl00_ContentPlaceHolder1_txtToolInput").Value = "mx:domain.co.uk"
    $ie.Document.getElementById("ctl00_ContentPlaceHolder1_btnAction").Click()
    Start-Sleep -Seconds 10
    $ie.Document.body.innerHTML | Out-File "C:\NETESP\MXRecords\MXRecordsHTML.txt" -Encoding ASCII
    $ie.Quit()

Is there something i need to have installed on a server 2008 box to make this return a value?

Thanks in Advance

Can you access the URL from an IE user session? Security policy allow navigation? — CB.
– CB., Commented Sep 17, 2012 at 9:16
Hi Christian, i can access the webpage on the server, the script is also able to submit the value and click the lookup button to retrieve the results. It's when i try to get the HTMl at the end that the issue occurs. — steveo448
– steveo448, Commented Sep 17, 2012 at 9:22
I've also set $ie.Visible = $true in place of $ie.silent = $true but this made no difference. — steveo448
– steveo448, Commented Sep 17, 2012 at 9:26
I can tell that I've tested your code in a W2008 R2 server and it works.... — CB.
– CB., Commented Sep 17, 2012 at 9:37

Shay Levy · Accepted Answer · 2012-09-17 09:38:33Z

3

Could be a timing issue? I can't test on 2008 but give this a try:

$ie = New-Object -com InternetExplorer.Application
$ie.silent = $false
$ie.navigate2("http://www.mxtoolbox.com/")
while($ie.busy) {start-sleep 1}
$ie.Document.getElementById("ctl00_ContentPlaceHolder1_txtToolInput").Value = "mx:domain.co.uk"
$ie.Document.getElementById("ctl00_ContentPlaceHolder1_btnAction").Click()

# wait for the result page 
While($ie.LocationURL -eq 'http://www.mxtoolbox.com/') {
    Write-Warning "Waiting for result"
    Start-sleep 1
}

# grab the table html
$table = $ie.Document.getElementsByTagName('TABLE') | Where-Object {$_.className -eq 'table table-striped table-bordered table-condensed tool-result-table'}
$table.outerHTML | Out-File "C:\NETESP\MXRecords\MXRecordsHTML.txt" -Encoding ASCII
$ie.Quit()

answered Sep 17, 2012 at 9:38

Shay Levy

128k34 gold badges193 silver badges210 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

steveo448 Over a year ago

Hi Shay, Thanks for helping on this one. The code you've given didn't directly work but your thought did lead me down a different path, i've now been able to get the HTML from just using a div Id further up that contains what i want instead of the whole page.

Collectives™ on Stack Overflow

Get HTML from website using DOM

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related