How to exit a powershell script with exitcode after using Write-Error?

I noticed that if I use Write-Error and after that exit 123 the $lastexitcode variable is not changed, instead it still contains the exit code from the previous command.

Given I have these files:

test-out.ps1:

Write-Output "hello"
exit 3

test-err.ps1

Write-Error "hello"
exit 123

Then I call in a powershell:

.\test-out.ps1
// displays: hello
$lastexitcode
// displays: 3

.\test-err.ps1
// displays: Write-Error: hello
$lastexitcode
// also/still displays: 3

I expected $lastexitcode after .\test-err.ps1 to be 123.

My workaround is to use [Console]::Error.WriteLine("hello"), but it seems like Write-Error should be the preferred way of doing this.

The documentation says

To write a non-terminating error, enter an error message string, an ErrorRecord object, or an Exception object. Use the other parameters of Write-Error to populate the error record.

It doesn't mention that it will prevent setting a custom exit code. And worse, using this doesn't set an exit code at all. If it was 0 before, then after using Write-Error and then exit 1 won't even work.

Am I missing something?