Speed up powershell output

Question

I'm running a query to pull data from AD. I have a 2 line script to get the data. The first line takes 4 or 5 minutes to complete. It returns around 200,000 users. The second line takes about 20 hours to complete.

$allEnabled = Get-adUser -f { (userAccountControl  -eq 512) -and (Enabled -eq $true) }   -properties employeeID, emailaddress, GivenName, SurName 
$allEnabled | ft employeeID, emailaddress, GivenName, SurName | out-file d:\temp\output.txt

I've tried writing the output one line at a time, and I've tried stuffing it in an array but I can't find a way to write the data any quicker. I've run it from a beefy server with ample resources so CPU and memory are not constrained.

Firstly, every time you use | (pipe), you are slowing yourself down but that's probably not the real reason for slowness. When PowerShell cmdlets lets you down, your best bet is to use .NET methods directly. — Adil Hindistan
– Adil Hindistan, Commented Nov 11, 2015 at 22:34
@AdilHindistan You are using the pipe whether you explicitly type it out or not. The pipe can streamline many actions. — Matt
– Matt, Commented Nov 11, 2015 at 22:45
@Matt Not true, but this is not a good place to argue the point either. Pipeline has its uses but should in general be avoided when you have RAM and need speed. — Adil Hindistan
– Adil Hindistan, Commented Nov 11, 2015 at 22:49

Matt · Accepted Answer · 2015-11-11 22:43:30Z

3

Not sure of the gains here but you can make some simple changes... Namely avoiding saving the results in a variable and not using the Format- cmdlets for output (that is not to console!). In the case of the latter I think it needs to enumerate the entire object so it know how to display it on console.

Since you are using object you would be better off taking advantage of that and using Export-CSV for output.

Get-adUser -f { (userAccountControl  -eq 512) -and (Enabled -eq $true) }   -properties employeeID, emailaddress, GivenName, SurName | 
    Select employeeID, emailaddress, GivenName, SurName | 
    Export-CSV -NoTypeInformation d:\temp\output.txt

This should effectively be writing one object at a time to file. Format-Table I think was the biggest issue here.

answered Nov 11, 2015 at 22:43

Matt

47k9 gold badges90 silver badges125 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

dft277 Over a year ago

Using this method the entire process completed in under 3 minutes. Much more efficient that what I was doing.

Adil Hindistan · Accepted Answer · 2015-11-11 22:45:10Z

1

Firstly, every time you use | (pipe), you are slowing yourself down but that's probably not the real reason for slowness. When PowerShell cmdlets lets you down, your best bet is to use .NET methods directly. You could for example try StreamWriter:

$sw = [System.IO.StreamWriter] "d:\temp\output.txt"
$allEnabled = Get-adUser -f { (userAccountControl  -eq 512) -and (Enabled -eq $true) }   -properties employeeID, emailaddress, GivenName, SurName
$allEnabled | % { $sw.WriteLine("$($_.employeeID), $($_.emailaddress), $($_.GivenName), $($_.SurName)") }
$sw.close()

answered Nov 11, 2015 at 22:45

Adil Hindistan

6,7254 gold badges29 silver badges28 bronze badges

3 Comments

sodawillow Over a year ago

With 1000 users this one seems a bit faster than the 2 other answers. I'm tempted to add 50k fake accounts and re-run ^_^

dft277 Over a year ago

Using this method the entire process completed in under 3 minutes. Much more efficient that what I was doing.

Matt Over a year ago

@dft277 Streamwriter will beat most native cmdlets (if not all) hands down.

Kirill Pashkov · Accepted Answer · 2015-11-12 11:35:55Z

0

Format-Table will not do any form of result formatting. It is only used for console outputs. You want to pipe out data directly to file in your case.

Also I suggest you to use LDAP filter instead. For my system it works slighty faster than -Filter parameter.

Get-ADUser -LDAPFilter "(&(objectCategory=person)(objectClass=user)(!userAccountControl:1.2.840.113556.1.4.803:=2)(userAccountControl=512))" -properties employeeID, emailaddress, GivenName, SurName | Export-Csv -Path c:\adusers.csv -NoTypeInformation

answered Nov 12, 2015 at 11:35

Kirill Pashkov

3,2461 gold badge19 silver badges22 bronze badges

Collectives™ on Stack Overflow

Speed up powershell output

3 Answers 3

1 Comment

3 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related