Encoding problems when concatenating with Powershell

Question

Out of a bunch of files I want to concatenate all files whose filename starts with a 1 into a textfile named 1.txt. The encoding of the source files is UTF16-LE and this shall also be the encoding of the target file. But using my powershell script (code below) results in a UTF-8 file with defect special signs (Umlauts etc.)

This is my code:

Powershell.exe cat  Disc:\data\files\1\filename-1* > Path:\here\1.txt

I found several approaches using Get-Content and Out-File but was not able to adapt those commands to my wildcard needs. As far as I understand, cat is a standard alias of get-content, so I wonder why I cannot just replace cat with get-content. Additionally, get-content has a -Encoding CharSet parameter but changing my code like this:

Powershell.exe cat  Disc:\data\files\1\filename-1* > Path:\here\1.txt -encode Unicode

(which is the charset for UTF-16 format little-endian byte order) does not change a thing. Neither does BigEndianUnicode or UTF8.

Thanks for any help!

Ansgar Wiechers · Accepted Answer · 2014-12-01 09:35:57Z

3

You can't specify the encoding when using a redirection operator (>). Use the Out-File cmdlet instead:

Get-Content C:\path\to\input* | Out-File D:\path\to\output.txt -Encoding Unicode

answered Dec 1, 2014 at 9:35

Ansgar Wiechers

202k27 gold badges286 silver badges363 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

Largo Over a year ago

Thank you - this is good to know! But somehow, your code does not work as well. Here is what I did: 'Powershell.exe Get-Content J:\path\content\name1* -Encoding Unicode | Powershell.exe Out-File J:\newpath\1.txt -Encoding Unicode' I tried with and without Powershell.exe after the pipe but it seems to be necessary. And I tried with only one -Encoding as in your code as well. But the only result I get is an empty file 1.txt

Ansgar Wiechers Over a year ago

@Daniel Where exactly in my answer do you see a Get-Content -Encoding Unicode, or a powershell.exe? The code is meant to be run from a PowerShell prompt. If you want to run it from somewhere else (a command prompt for instance) you need to run it like this: powershell.exe -Command "& { Get-Content ... }"

Micky Balladelli Over a year ago

@Daniel maybe you can provide a sample of the text you are trying to read and save, so we can see. I tried Ansgar's code with Japanese and it worked.

Largo Over a year ago

@AnsgarWiechers Sorry, I thought I have mentioned my intention to use the code in a batch file ;) After editing it like you said it finally worked out, thanks alot.

Collectives™ on Stack Overflow

Encoding problems when concatenating with Powershell

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related