Array causing 'system.outofmemoryexception'

Question

I am running the below script and it is causing exception of type 'system.outofmemoryexception' was thrown

I believe that it is due to the @Results array growing past the 2gb allocated to windows shell. Is there possibly a way to iterate through the results, or am I stuck with allocating more memory (which could ultimately be a lot...)?

$Path = "path to output" 

### Get all PF
$publicFolders = Get-PublicFolder "Public Folder Name" -Recurse -resultsize unlimited | Select-Object *


### Array to contain results
$results = @()

###Begin looping through each PF and grab each user/group with AccessRights to that folder
$final = ForEach($pf in $publicFolders){    
    $perms     = Get-PublicFolderClientPermission -Identity $pf.Identity | Where-Object {$_.User -notmatch "Default|Anonymous|S-X-X-XX"} 
    
    Foreach($perm in $perms){
        $temp = [PSCustomObject]@{
                    MailFolderName = $pf.Identity 
                    UserWithPerms  = $perm.User
                    AccessRights   = $perm | Select-Object -ExpandProperty AccessRights

                }
                $results += $temp
    }
                   
}

$final | Export-Csv $path -NoTypeInformation

Am I barking up the wrong tree?

Thanks in advance.

Mathias' solution is preferable, but what likely contributed to your problem: Incrementally extending arrays in a loop with += is inefficient, because a new array must be created behind the scenes in every iteration, because arrays are of fixed size; a much more efficient approach is to use a foreach loop as an expression and let PowerShell itself collect the outputs in an array: [array] $outputs = foreach (...) { ... } - see this answer. — mklement0
– mklement0, Commented Oct 8, 2021 at 16:17

Mathias R. Jessen · Accepted Answer · 2021-10-08 16:10:42Z

3

Use the ForEach-Object cmdlet instead of a foreach(){} loop statement for the outer loop - this way you can start piping the output to Export-Csv immediately instead of buffering it in an array:

$publicFolders |ForEach-Object {
    $pf    = $_
    $perms = Get-PublicFolderClientPermission -Identity $pf.Identity | Where-Object {$_.User -notmatch "Default|Anonymous|S-X-X-XX"} 
    
    Foreach($perm in $perms){
        [PSCustomObject]@{
            MailFolderName = $pf.Identity 
            UserWithPerms  = $perm.User
            AccessRights   = $perm | Select-Object -ExpandProperty AccessRights
        }
    }                   
} | Export-Csv $path -NoTypeInformation

Alternatively, flush the partial set of results to file after enumerating the permissions for each folder:

ForEach($pf in $publicFolders){    
    $perms     = Get-PublicFolderClientPermission -Identity $pf.Identity | Where-Object {$_.User -notmatch "Default|Anonymous|S-X-X-XX"} 
    
    $results = Foreach($perm in $perms){
        [PSCustomObject]@{
            MailFolderName = $pf.Identity 
            UserWithPerms  = $perm.User
            AccessRights   = $perm | Select-Object -ExpandProperty AccessRights
        }
    }

    # flush the buffer before moving to the next set of permissions
    $results |Export-Csv $path -NoTypeInformation -Append
}

answered Oct 8, 2021 at 16:10

Mathias R. Jessen

178k13 gold badges175 silver badges234 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Techknow Over a year ago

Outstanding! So simple. Obviously, it worked. I need to get down into the weeds re: forEach vs forEach-object (among other things), but you Sir are genius. Thanks mate. As you slide down the bannister of life, may all the splinters point in the right direction.

Mathias R. Jessen Over a year ago

@MarcEverlove Most excellent to hear, and you're very welcome! :) The short of it is that foreach(){...} is generally "low-overhead" - less memory and usually slightly faster processing time - but it doesn't support writing output to the pipeline. ForEach-Object does write to the pipeline, but it doesn't handle flow control statements (break, continue) very well, incurs higher memory cost and takes an O(N) perf hit from input pipeline binding. ForEach-Object also does a bunch of things foreach(){} doesn't, but that's part of the longer version of it :)

Collectives™ on Stack Overflow

Array causing 'system.outofmemoryexception'

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related