When processing CSV input with a ForEach-Object loop you need to output the data back to the pipeline. Also, the -replace operator doesn't modify variables or properties in-place. It takes the value, does the work, and outputs the modified string to the success output stream. If you want to update a property you need to assign the modified value back to that property.
Change this:
$TestFile = Import-Csv .\file.csv
$NewFile = $TestFile | ForEach-Object {$_."Username" -replace 'domain\\',''}
into this:
$NewFile = Import-Csv .\file.csv | ForEach-Object {
$_.Username = $_.Username -replace 'domain\\', '' # update username
$_ # feed data back into the pipeline
}
and the code will do what you want.