Bigger txt file with less rows on Windows after Powershell Select-String

Question

I applied several filters to a text files using Powershell Get-Content and the -nomatch operator and then i spooled the result to a file.

gc file.txt | {?_ -notmatch 'excl1|excl2|excl3'} | out-file newfile.txt

What happens is that the output file (newfile.txt) has less lines, but it is reported by windows with a bigger size than file.txt.

Has someone ever encountered this behavior? How can I have the correct size reported by windows? I checked the number of rows, the file with less rows is reported as bigger in size.

filter a text file (.txt) using several keywords in the format: gc file.txt | ?{$_ -notmatch 'excl1|excl2|excl2'} | out-file newfile.txt. newfile.txt is bigger than file.txt — bidi
– bidi, Commented Dec 21, 2015 at 17:26
Give us an example file with your comparing file sizes. .... Also need to know the encoding. I am guessing that is where the issue comes from? — Matt
– Matt, Commented Dec 21, 2015 at 17:58
I agree with Matt. The output file is probably larger because Out-File creates the file with Unicode encoding (2 bytes per character) whereas the input file most likely is ASCII encoded (1 byte per character). — Ansgar Wiechers
– Ansgar Wiechers, Commented Dec 21, 2015 at 18:10

Community · Accepted Answer · 2020-06-20 09:12:55Z

3

I'm certain you have an encoding issue. By default Get-Content uses ascii whereas Out-File uses Unicode.

From TechNet

-Encoding

Specifies the type of character encoding used in the file. Valid values are "Unicode", "UTF7", "UTF8", "UTF32", "ASCII", "BigEndianUnicode", "Default", and "OEM". "Unicode" is the default.

Use -Enconding ascii with Out-File or just use Set-Content as it is the partner of Get-Content.

Get-Content file.txt | {?_ -notmatch 'excl1|excl2|excl3'} | 
    out-file -Encoding ascii newfile.txt
    # or
    Set-Content newfile.txt

Coming from the other direction if you have issues with your input file Get-Content in PowerShell v3 and above also supports -Enconding

edited Jun 20, 2020 at 9:12

CommunityBot

11 silver badge

answered Dec 21, 2015 at 18:21

Matt

47k9 gold badges90 silver badges125 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Bigger txt file with less rows on Windows after Powershell Select-String

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related