high memory consumption in powershell foreach loop

Question

I am trying to write a powershell script that will take users from our oracle DB export (CSV) and use that to either update info in Active Directory or create new accounts (with the Quest AD cmdlets, set-qaduser). The script I have is working, however it will not finish the foreach loop because it is running out of memory. The CSV has about 1,300 lines to loop through and the box has 12GB of ram.

I think there is an issue with my foreach loop and just processing it the most efficient way, so that is where I am looking for help. Script is below:

Add-PSSnapin Quest.Activeroles.ADManagement

Import-Csv \\pathtocsv\importusers.csv | foreach {

[string]$upn=$_.FIRST_NAME[0]+$_.LAST_NAME+"."+$_.ASSOC+"@innout.corp"

#check to see if the AD account already exists, if not, create it
if (!(get-qaduser $upn))
{

#because there are some blank/null values for phone numbers we need to only put in the variable values that have data, otherwise the script will bomb out
if($_.HOME){
$homephone=$_.home}
else{
$homephone=" "}

if($_.CELL){
$cellphone=$_.cell}
else{
$cellphone=" "}

$mgr=Get-QADUser -IncludedProperties employeeid -oa @{employeeid=$_.mgr}

#Object attribute hashtable, ADattribute and the value you want to put.

$oa=@{
Department=$_.ctr_name;
division=$_.division;
employeeid=$_.assoc;
ExtensionAttribute10=$_.mgr;
ExtensionAttribute11=$_.region_name;
ExtensionAttribute12=$_.hire_date;
ExtensionAttribute13=$_.dob;
ExtensionAttribute14=$_.region;
ExtensionAttribute15=$_.mgr_name;
DepartmentNumber=$_.ctr
}

New-QADUser -ParentContainer "OU=StoreManagers,OU=Stores,DC=contoso,DC=com" -Name $_.full_name -title $_.title -DisplayName $_.full_name -firstname $_.first_name -lastname $_.last_name -upn $upn -homephone $homephone -mobilephone $cellphone -manager $mgr -telephonenumber $_.work -ObjectAttributes $oa

}

#this else statement is if the AD account already exists, then just come here and update the account.
else

{

if($_.HOME){
$homephone=$_.home}
else{
$homephone=" "}

if($_.CELL){
$cellphone=$_.cell}
else{
$cellphone=" "}

$mgr=Get-QADUser -IncludedProperties employeeid -oa @{employeeid=$_.mgr}

$oa=@{
Department=$_.ctr_name;
division=$_.division;
employeeid=$_.assoc;
ExtensionAttribute10=$_.mgr;
ExtensionAttribute11=$_.region_name;
ExtensionAttribute12=$_.hire_date;
ExtensionAttribute13=$_.dob;
ExtensionAttribute14=$_.region;
ExtensionAttribute15=$_.mgr_name;
DepartmentNumber=$_.ctr
}

set-qaduser -identity $upn -DisplayName $_.full_name -firstname $_.first_name -lastname $_.last_name -title $_.title -homephone $homephone -mobilephone $cellphone -manager $mgr -telephonenumber $_.work -ObjectAttributes $oa 

}
}

#This section will disable/move/delete managers that have left the company or stepped down to a non-managment role. 

$deletedusers=Import-Csv \\pathtocsv\importusers.csv
foreach ($deleteduser in $deletedusers) {
[string]$samdelete=$deleteduser.FIRST_NAME[0]+$deleteduser.LAST_NAME+"."+$deleteduser.ASSOC
Disable-QADUser $samdelete
Move-QADObject $samdelete -NewParentContainer "OU=ToBeDeleted,OU=StoreManagers,OU=Stores,DC=contoso,DC=com"
set-qaduser $samdelete 
}

#This section sets all the DM Division numbers

$dmusers=Import-Csv \\pathtocsv\importusers.csv

foreach ($dmuser in $dmusers) {
$oa=@{
division=$dmuser.division
}
Get-QADUser -oa @{employeeid=$dmuser.assoc} | set-qaduser -oa $oa
}

Did you try adding -DontUseDefaultIncludedProperties to Get-QADUser calls? — Micky Balladelli
– Micky Balladelli, Commented Nov 18, 2014 at 18:57
Thanks Micky. I did attempt to use -DontUseDefaultIncludedProperties and then just included all the properties that I needed to set, but it still grows. I think there may be a memory leak in Quest cmdlets. see comment below. — INOBrad777
– INOBrad777, Commented Nov 19, 2014 at 22:40

Community · Accepted Answer · 2017-05-23 12:05:46Z

0

This sounds like a memory leak. There is another a question about XML manipulation that manifests similar situation. The solution is upgrade to Powershell 3+. Some work-around might be possible with adding remove-variable and [GC]::Collect() statements to the loop in order to free resources.

In addition, a blog article describes Quest AD cmdlet resource use and has some tips too. The main caveat is that Quest cmdlets easily retrieve all the AD users, wrap data as Powershell objects - and that consumes lots of memory.

edited May 23, 2017 at 12:05

CommunityBot

11 silver badge

answered Nov 18, 2014 at 19:05

vonPryz

24.3k10 gold badges58 silver badges69 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

INOBrad777 Over a year ago

Thanks vonPryz. I tried to enable the Garbage Collector and remove the variables, but it is still growing (about 3MB per get-qaduser lookup). If I just run get-qaduser from powershell (outside the script) the memory for powershell.exe goes up by 3MB and does not release. Memory leak in the Quest cmdlets?

INOBrad777 Over a year ago

Ohh, and I did update to Powershell 4 just now to make sure I was running the most up to date shell, just in case they fixed some memory leak.

INOBrad777 · Accepted Answer · 2014-11-20 17:49:08Z

So it turns out there is a memory leak (at least in the version I am running of the Quest cmdlets). I ran (outside of the script) just a get-qaduser on my user account and I saw the powershell.exe process grow by 6MB. I then ran it again and again, and it would grow by 6MB each time and never release the memory. I ended up converting the above script to use the built-in set-aduser and it ran only using 50MB total for the whole script duration.

On a side note, I did attempt to look for updated versions of the quest cmdlets, but are they gone now? I know Dell bought them, but you can only purchase the Active Roles Server from Dell, no more free quest.activeroles.admanagement? If anyone knows where to get updated Quest for Active Directory cmdlets, please let me know.

Collectives™ on Stack Overflow

high memory consumption in powershell foreach loop

2 Answers 2

2 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related