Handle ASCII output when executing external exe within powershell

Question

When invoking AWS CLI executable from within powershell (v5.1), special characters are not handled properly by powershell. Here is an example:

# This works: powershell is forwarding the output directly to stdout without touching it
> aws cognito-idp describe-user-pool --user-pool-id $MyPoolId
{
    "UserPool": {
        [...]
        "EmailVerificationMessage": "Votre code de vérification est {####}."
        [...]
    }
}

# This doesn't work: powershell handles the command's output encoding improperly
> aws cognito-idp describe-user-pool --user-pool-id $MyPoolId | Out-Host
{
    "UserPool": {
        [...]
        "EmailVerificationMessage": "Votre code de vÚrification est {####}."
        [...]
    }
}

The problem is exactly the same when you want to save the output to a variable ($foo = aws ...) or redirect it to a file (| Out-File "foo.json" or > "foo.json"). Changing the output encoding (| Out-File "foo.json" -Encoding xxx) does not solve the issue, because the parsing error is made when powershell decodes the exe output, not when it outputs it to the file. The issue is not present when doing the same thing from CMD.

We could switch to the AWS tools for powershell (which should handle encoding properly) instead of using the standard AWS CLI, but I was wondering how you would solve this issue when there is no powershell alternative.

Decoding output from an exe is managed by [Console]::OutputEncoding. Decoding output to be sent to an exe is managed by $OutputEncoding. Does changing those to the correct encoding help the situation? — AdminOfThings
– AdminOfThings, Commented Oct 16, 2019 at 10:27

Maxime Rossini · Accepted Answer · 2019-10-16 14:42:04Z

As mentioned by @AdminOfThings in this comment, the encoding used by powershell to decode the output from an exe is managed by [Console]::OutputEncoding (whereas $OutputEncoding manages the the encoding used to send data to an exe).

The value of this variable in my case was ibm850 :

> [Console]::OutputEncoding

IsSingleByte      : True
BodyName          : ibm850
EncodingName      : Europe de l'Ouest (DOS)
HeaderName        : ibm850
WebName           : ibm850
WindowsCodePage   : 1252
IsBrowserDisplay  : False
IsBrowserSave     : False
IsMailNewsDisplay : False
IsMailNewsSave    : False
EncoderFallback   : System.Text.InternalEncoderBestFitFallback
DecoderFallback   : System.Text.InternalDecoderBestFitFallback
IsReadOnly        : False
CodePage          : 850

Switching to iso-8859-1 solved my issue:

> [Console]::OutputEncoding = [System.Text.Encoding]::Default
> [Console]::OutputEncoding

IsSingleByte      : True
BodyName          : iso-8859-1
EncodingName      : Europe de l'Ouest (Windows)
HeaderName        : Windows-1252
WebName           : Windows-1252
WindowsCodePage   : 1252
IsBrowserDisplay  : True
IsBrowserSave     : True
IsMailNewsDisplay : True
IsMailNewsSave    : True
EncoderFallback   : System.Text.InternalEncoderBestFitFallback
DecoderFallback   : System.Text.InternalDecoderBestFitFallback
IsReadOnly        : True
CodePage          : 1252

Then:

> aws cognito-idp describe-user-pool --user-pool-id $MyPoolId | Out-Host
{
    "UserPool": {
        [...]
        "EmailVerificationMessage": "Votre code de vérification est {####}."
        [...]
    }
}

Collectives™ on Stack Overflow

Handle ASCII output when executing external exe within powershell

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related