You need to go the other way around - capture the first two letters, and the last letter, and replace with them separated by **:
var name ="Jason Mamoa";
var regex =/\b(\w{2})\w+(\w)\b/g;
console.log(name.replace(regex, '$1**$2'));
If you want the resulting string to be the same length as the original, use a replacer function:
var name ="Jason Mammmmoa";
var regex =/\b(\w{2})(\w+)(\w)\b/g;
console.log(name.replace(
regex,
(_, first, middle, last) => `${first}${'*'.repeat(middle.length)}${last}`
));
Note that [^\W] is equivalent to \w.
To also mask the last 2 characters of 3 and 4 character names, capture up to 2 leading characters, and have the last be optional:
var name ="Eli Mose";
var regex =/\b(\w{1,2})(\w{2,}?)(\w?)\b/g;
console.log(name.replace(
regex,
(_, first, middle, last) => `${first}${'*'.repeat(middle.length)}${last}`
));