How to pass a custom function inside a ForEach-Object -Parallel

I added a whole set of custom functions to parallel processes via a ps1 file by using an include inside the loop. This keeps things very clean and neat.

ForEach-Object -Parallel {
    # Include custom functions inside parallel scope
    . $using:PSScriptRoot\CustomFunctions.ps1
    # Now you can reference any function defined in the file
    My-CustomFunction
    ....

This indeed incurs overhead requiring the loading of functions in each parallel process, but in my case this was miniscule related to the overall processing time.

So I figured out another little trick that may be useful for people trying to add the functions dynamically, particularly if you might not know the name of it beforehand, such as when the functions are in an array.

# Store the current function list in a variable
$initialFunctions=Get-ChildItem Function:

# Source all .ps1 files in the current folder and all subfolders
Get-ChildItem . -Recurse | Where-Object { $_.Name -like '*.ps1' } |
     ForEach-Object { . "$($_.FullName)" }

# Get only the functions that were added above, and store them in an array
$functions = @()
Compare-Object $initialFunctions (Get-ChildItem Function:) -PassThru |
    ForEach-Object { $functions = @($functions) + @($_) }

1..3 | ForEach-Object -Parallel {
    # Pull the $functions array from the outer scope and set each function
    # to its definition
    $using:functions | ForEach-Object {
        Set-Content "Function:$($_.Name)" -Value $_.Definition
    }
    # Call one of the functions in the sourced .ps1 files by name
    SourcedFunction $_
}

The main "trick" of this is using Set-Content with Function: plus the function name, since PowerShell essentially treats each entry of Function: as a path.

This makes sense when you consider the output of Get-PSDrive. Since each of those entries can be used as a "Drive" in the same way (i.e., with the colon).

I just figured out another way using get-command, which works with the call operator. $a ends up being a FunctionInfo object.

EDIT: I'm told this isn't thread safe, but I don't understand why.

function hi { 'hi' }
$a = get-command hi
1..3 | foreach -parallel { & $using:a }

hi
hi
hi

This might be a more elegant low-code option to get a function into Foreach-Object -Parallel block:

$m = New-Module -Name MyFunctions -ScriptBlock {
    function Func-Timestamp {
        return [DateTime]::Now.ToString("HH:mm:ss.ff")
    }
}
    
$files | ForEach-Object -Parallel {
    import-module $using:m -DisableNameChecking
    [Console]::WriteLine("Here is the Time! $(Func-TimeStamp)")
}

You create $m to be module variable and then import it into the Foreach-Object loop.

Refer to:

1. https://stackoverflow.com/a/61273544
2. https://stackoverflow.com/a/77054556
3. https://superuser.com/questions/1767655/how-to-import-a-module-in-powershell-from-a-string-variable

The method with the smallest change is:

$scriptBlock = {
    function Get-Custom {
        Param ($A)
        "[$A]"
    }
}

$scriptBlockString = $scriptBlock.ToString()

if ($PSVersionTable.PSVersion.Major -ge 7) {
    "Apple", "Banana", "Grape"  | ForEach-Object -Parallel {
        $scriptBlockFromString = [scriptblock]::Create($using:scriptBlockString)
        New-Module -ScriptBlock $scriptBlockFromString -Name MyCustomModule | Import-Module
        Get-Custom $_
    }
}
else {
    # If your script also need to running in PowerShell 5
    "Apple", "Banana", "Grape"  | ForEach-Object -Parallel {
        $scriptBlockFromString = [scriptblock]::Create($scriptBlockString)
        New-Module -ScriptBlock $scriptBlockFromString -Name MyCustomModule | Import-Module
        Get-Custom $_
    }
}

This example expands on the original question with explanation about code safety and performance.

The simplest approach is to pass the function as a string (this was answered correctly), however I feel the audience would benefit from understanding code safety implications of code injection in this way.

The example below uses powershell modules to provide a safer alternative and expands on the original correct answer showing how to also pass common parameters like '-Verbose'.

# Parallel Function scope-test.psm1
#   demonstrates different ways to handle function scope in parallel runspaces
  # reference: https://devblogs.microsoft.com/powershell/powershell-7-0-parallel-foreach-object/
# Various approaches are demonstrated including:
#   1. Failing approach where the function is not in scope
#   2. Common approach where the function is passed as a string and recreated in the parallel runspace
#   3. Safe approach where the module is imported in the parallel runspace
#   4. Clean approach where the module is imported using the 'using module' statement

# initialization
$module = $PSScriptRoot + '\scope-test.psm1'

function Start-Work {
    # Function to be used in parallel runspaces
    [CmdletBinding()]
    Param([ValidateNotNullOrEmpty()][string] $message = 'starting work')
    Write-Host $message
    Write-Verbose "Work started at $(Get-Date -Format o)"
    Start-Sleep -Seconds 1 # simulate work
}

# similar to the ToString() approach but is more explicit
$functionAsString = (Get-ChildItem function:Start-Work).Definition

function Test-ParallelFails {
    # approach does not work because Start-Work function is out of scope in the parallel runspace
    1..1 | ForEach-Object -Parallel {
        Start-Work 'example that fails!'
    }
}

function Test-ParallelByPassingFunctionAsString {
    # Common recommended approach that recreates the required function in the parallel runspace 
    #  using serialized script passed in a variable
    # Unsafe because it allows for code injection if the variable is not controlled and
    #   is hard to validate for mitigation
    1..9 | ForEach-Object -ThrottleLimit 3 -Parallel {
        New-Item -Path Function:Start-Work -Value $using:functionAsString -Force | Out-Null
        Start-Work "$_ : example where the function is passed using a string and recreated in the new runspace"
    }
}

function Test-ParallelModule {
    [CmdletBinding()] param ()
    # Safe approach that imports the specified module into the parallel runspace, 
    #   avoids code injection issues and verifies the module path.
    #   requires the module to be in a file or well-formed module folder
    # This example also shows throttling at 3 threads and how to pass-through verbose 
    #   preference (or other common parameters)
    # This approach is also compatible with -AsJob if needed
    1..9 | ForEach-Object -ThrottleLimit 3 -Parallel {
        $modulePath = Resolve-Path $using:module
        Import-Module $modulePath -Verbose:$using:VerbosePreference
        Start-Work "$_ : example that uses import-module in the new runspace" -Verbose:$using:VerbosePreference
    }
}

function Test-ParallelUsingModule {
    # Clean approach that also works with classes, dotnet types and 
    #   works by importing the module and any types into the parallel runspace
    #   requires the module to be in a file or well-formed module folder
    #   using module - has a requirement to be the first statement in the script block
    #                  and this is why invoke-expression is used here
    # Safer than recreating the function because input validation is used to mitigate code injection 
    #   when using invoke-expression
    # This approach is also compatible with -AsJob if needed
    1..9 | ForEach-Object -ThrottleLimit 3 -Parallel {
        $modulePath = Resolve-Path $using:module
        Invoke-Expression "using module $modulePath"
        Start-Work " $_ : example that uses 'using module' to import module, classes and types in the new runspace"
    }
}

function Test-AllWithMeasureCommand {
    # This function runs all 3 working approaches and measures the time taken for each
    #   tl;dr all approaches are similar in initialization performance and insignificant for most use cases
    Measure-Command {
        Test-ParallelByPassingFunctionAsString
    }
    Measure-Command {
        Test-ParallelModule
    }
    Measure-Command {
        Test-ParallelUsingModule
    }
}

Export-ModuleMember -Function Test-ParallelFails, Test-ParallelByPassingFunctionAsString, Test-ParallelUsingModule, Test-ParallelModule, Test-AllWithMeasureCommand, Start-Work