Powershell's -replace operator is based on regular expressions. And since . is a wildcard in regex, what should be happening is that each character in the file name is being replaced with the resulting string. So test.txt would become -b.-b.-b.-b.-b.-b.-b in your example.
You likely want to use the Replace method of the .NET String type like this instead.
dir | Rename-Item -NewName { $_.Name.Replace('.','-b.') }
If you want to keep using -replace, you need to escape the . in your expression like this.
dir | Rename-Item -NewName { $_.Name -replace '\.','-b.' }
Both of these have a couple edge case problems that you may want to avoid. The first is narrowing the scope of your dir (which is just an alias for Get-ChildItem) to avoid including files or directories you don't actually want to rename. The second is that a simple replace in the file name doesn't account for file names that contain multiple dots. So you may want to ultimately do something like this if you only care about SVG files that may have multiple dots.
Get-ChildItem *.svg -File | Rename-Item -NewName { "$($_.BaseName)-b$($_.Extension)" }