Powershell v2.0 Using multiple threads

Question

Basic script idea: Hello. I've created a powershell script which I use to check the filesizes of certain executables, and then keep them in a text file. Next time the script runs, if a filesize differs it will replace the one in the text file with the new one.

The structure: I have a main script and a folder which contains many scripts, each for every executable of which I want to check the filesize. So the scripts in the folder will return a string containing the link to the executable, which will be fed to the main script.

The code:

$progdir = "C:\script\programms"
$items = Get-ChildItem -filter *.ps1 -Path $progdir
$webclient = New-Object System.Net.WebClient

$filesizes = get-content C:\updatechecker\programms\filesizes
if ($filesizes.length -ne $items.length) { 
    if ($filesizes.length -eq $null) {
        Write-Host ("Building filesize database...") -nonewline
    } 
    else { 
        Write-Host ("Rebuilding filesize database...") -nonewline 
    }
    clear-content  C:\programms\filesizes

    for ($i=0; $i -le $items.length-1; $i++) {
        $command = "c:\programms\" + $items[$i].name
        $link = & $command
        $webclient.OpenRead($link) | Out-Null
        $filesize = $webclient.ResponseHeaders["Content-Length"]
        $filesize >> C:\programms\filesizes
    }
    echo "Done." 
} 
else {
    ...

Question: This for loop is the one I want to run in parallel. I need your advice on how to do this since I'm new to powershell. I tried to implement a few things I found but they didn't work correctly (took very long to finish, output errors, multiple entries of filesizes in my filesizes file). I suspect it's a synchronization issue and somehow I need to lock the critical parts. Isn't there anything like omp parallel for in powershell? :P

Any help,advice on how to achieve this would be appreciated :)

edit:

Get-Job | Remove-Job -Force
$progdir = "C:\programms"
$items = Get-ChildItem -filter *.ps1 -Path $progdir
$webclient = New-Object System.Net.WebClient
$filesizes = get-content C:\programms\filesizes

$jobWork = {
    param ($MyInput)
    $command = "c:\programms\" + $MyInput
    $link = & $command
    $webclient.OpenRead($link) | Out-Null
    $filesize = $webclient.ResponseHeaders["Content-Length"]
    $filesize >> C:\programms\filesizes

}
foreach ($item in $items) {

    Start-Job -ScriptBlock $jobWork -ArgumentList $item.name | out-null
}

Get-Job | Wait-Job
Get-Job | Receive-Job | Out-GridView | out-null
echo "Done."

Edit 2: Used code I found here: http://ryan.witschger.net/?p=22

$mutex = new-object -TypeName System.Threading.Mutex -ArgumentList $false, “RandomGlobalMutexName”;
$MaxThreads = 4
$SleepTimer = 500


$jobWork = {
    param ($MyInput)
    $webclient = New-Object System.Net.WebClient
    $command = "c:\programms\" + $MyInput
    $link = & $command
    $webclient.OpenRead($link) | Out-Null

    $result = $mutex.WaitOne();

    $file = $webclient.ResponseHeaders["Content-Length"]
    $file >> C:\programms\filesizes

    $mutex.ReleaseMutex();
}

$progdir = "C:\programms"
$items = Get-ChildItem -filter *.ps1 -Path $progdir
$webclient = New-Object System.Net.WebClient
$filesizes = get-content C:\programms\filesizes

Get-Job | Remove-Job -Force

$i = 0

ForEach ($item in $items){
    While ($(Get-Job -state running).count -ge $MaxThreads){

        Start-Sleep -Milliseconds $SleepTimer
    }

    $i++
    Start-Job -ScriptBlock $jobWork -ArgumentList $item.name | Out-Null


}

Community · Accepted Answer · 2017-05-23 12:13:40Z

3

You can run each iteration of the loop in a background job which is not the same a seperate thread in that it is a whole other PowerShell.exe process. Data is passed from the background processes through serialization.

To approach it using background jobs you'll need to define a script block that will do that actual work and then call the script block with parameters in each iteration of the loop. The script block can report back status via Write-Output or by throwing an exception.

You'll probably want to throttle how many concurrent background jobs are running. Here's an example of how to throttle:

$jobItems = "a", "b", "c", "d", "e"
$jobMax = 2
$jobs = @()

$jobWork = {
    param ($MyInput)
    if ($MyInput -eq "d") {
        throw "an example of an error"
    } else {
        write-output "Processed $MyInput"
    }
}

foreach ($jobItem in $jobItems) {
    if ($jobs.Count -le $jobMax) {
        $jobs += Start-Job -ScriptBlock $jobWork -ArgumentList $jobItem
    } else {
        $jobs | Wait-Job -Any
    }
}
$jobs | Wait-Job

As an alternative you might try eventing. Take a look at this thread for some examples of how to implement concurrency using events.

PowerShell: Runspace problem with DownloadFileAsync

You might be able to replace DownloadFileAsync with OpenReadAsync

edited May 23, 2017 at 12:13

CommunityBot

11 silver badge

answered Apr 16, 2012 at 19:38

Andy Arismendi

53.1k17 gold badges113 silver badges130 bronze badges

Sign up to request clarification or add additional context in comments.

10 Comments

kokotas Over a year ago

Thank you Andy. What I tried before was to create one more script which would pass my $items array to the main script using Start-Job. This though affected my whole script not just the for loop iterations, thus not working as I expected it to. I'll try what you suggest (calling script block withing the same script instead of calling the whole script file from separate script) and get back to you.

kokotas Over a year ago

I tried what you suggested andy but didn't work. Nothing is written on my file. But it I change $filesize >> C:\programms\filesizes to "something" >> C:\programms\filesizes, then I can see that a number of lines(something) will be written, a number which equals the $jobMax variable I set. I think it's a matter of locking critical parts really... I should mention that the scripts which are being called to return the link, they don't just do return "http://....", they also write to a file which is common for all of them. As soon as one is done the next overwrites the file and so on.

Andy Arismendi Over a year ago

@kokotas Without looking at your code I'm not sure what you need. However make sure to pass data to the script block via the -ArgumentList parameter on Start-Job. You will not be able to use variables in the scriptblock that you set outside the script block.

kokotas Over a year ago

I've edited my first post to reflect my progress. I understand I have to pass the $filesize variable via the -ArgumentList as well, correct?

kokotas Over a year ago

Actually no, I'm not using any other external variable other than $items. However when the scripts are being called $command = "c:\programms\" + $MyInput $link = & $command, to return the string link, they write to a temporary file that is common for all of them.

|

Collectives™ on Stack Overflow

Powershell v2.0 Using multiple threads

1 Answer 1

10 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

10 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related