Subversion status PowerShell non-ASCII characters problem

Question

I have a weird thing going on in my PowerShell script using Subversion commands. Following is an PowerShell example script:

    $svnOutput = svn status
    Write-Host "Output when saved in a variable"
    $svnOutput
    Write-Host "Direct Output"
    svn status

If I run this script (within the PowerShell console), I get two different outputs, if one of the files have non-ASCII-characters in the name (in my examples umlauts like üöä). This is the output:

    Output when saved in a variable
    ?    Test_���.txt
    Direct Output
    ?    Test_äöü.txt

I am working on a Windows Server 2022 with a VisualSVNServer version 5.4.1. I already tested following ideas:

    chcp 65001
    $svnOutput = svn status
    $svnOutput

    $OutputEncoding = [System.Text.Encoding]::UTF8
    [Console]::OutputEncoding = [System.Text.Encoding]::UTF8
    $svnOutput = svn status
    $svnOutput

    $svnOutput = & svn status | Out-String -Stream
    $svnOutput

    svn status > svn_status.txt
    $svnOutput = Get-Content -Path "svn_status.txt" -Encoding UTF8
    $svnOutput

    $svnOutput = & svn status | Out-String -Stream
    $svnOutput

But all of them give the same error.

PS: this also happens with other commands like

    $svnOutput = svn add . --force
    $svnOutput

which results in:

    A    Test_���.txt

Typing svn add . --force in a PowerShell instance, or even i a script works without any issues.

Try: $OutputEncoding = [Console]::InputEncoding = [Console]::OutputEncoding = New-Object System.Text.UTF8Encoding — iRon
– iRon, Commented Jan 30 at 17:26
Your first symptom is the inverse of what normally happens if [Console]::OutputEncoding doesn't match the actual output encoding: normally, direct-to-console output renders fine, but capturing in a variable reveals an encoding mismatch. Also, what do you mean by "in the normal powershell"? — mklement0
– mklement0, Commented Jan 30 at 18:02
@mklement0 by "in the normal powershell" I mean just typing the command in a powershell instance. But even when I type $svnOutput = svn status and afterwards output via typing $svnOutput in the powershell instace, I get the same sympton as via a script. — Sam
– Sam, Commented Jan 31 at 9:58
@iRon unfortunately, that doesn't work, neither in the shell nor in the script. — Sam
– Sam, Commented Jan 31 at 18:14
Glad to hear it helped, @Sam. Not restoring the original encoding is a problem if, later in the same session, you run a different console application that emits OEM-encoded output, because PowerShell will then misinterpret that — mklement0
– mklement0, Commented Feb 7 at 12:36

mklement0 · Accepted Answer · 2025-02-07 12:19:15Z

The SVN documentation states (emphasis added):

The default character encoding is derived from your operating system's native locale.

This is in the context of the --encoding parameter, which is documented as overriding the default encoding on submitting information ("your commit message"), but it seemingly (and sensibly) also applies when retrieving information.

On Windows, the native locale is the so-called legacy system locale, aka language for non-Unicode programs, and it determines two encodings, via Windows code pages: the OEM code page (wich may be, e.g., CP437 or CP850) - typically used by console (terminal) applications - and the ANSI code page (e.g., Windows-1252 or Windows-1251) - typically used by GUI applications.

While the SVN documentation doesn't spell out which of these two code pages the svn utility uses, per your own feedback it seems to be the system's active ANSI code page (which, as noted, is unusual, because console applications by convention use the OEM code page; python is similarly unusual).

PowerShell consoles on Windows use the OEM code page by default, as reflected in [Console]::OutputEncoding.

Thus, in order for PowerShell to interpret (decode) ANSI output correctly,^[1] [Console]::OutputEncoding must be (temporarily) set to the system's active ANSI code page, as follows:

& {
  # Temporarily change the expected output encoding to the ANSI code page.
  $prevEnc = [Console]::OutputEncoding
  [Console]::OutputEncoding = 
    [Text.Encoding]::GetEncoding(
      [int] (Get-ItemPropertyValue HKLM:\SYSTEM\CurrentControlSet\Control\Nls\CodePage ACP)
    )

  svn status

  # Restore the original encoding.
  Console]::OutputEncoding = $prevEnc

}

Note that (non-CJK) ANSI encodings are fixed single-byte encodings and therefore limited to 256 characters. If you need full Unicode support, use --encoding utf8 in your svn call and set [Console]::OutputEncoding = [Text.UTF8Encoding]::new()^[2]

Collectives™ on Stack Overflow

Subversion status PowerShell non-ASCII characters problem

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related