You can work with groups in a regex. You create groups by using parentheses in your regular expression. When you get a Match object, this object will contain a Group collection:
string input = "<html><img id=\"someImage\" src=\"C:\\logo.png\" height=\"64\" width=\"104\" alt=\"myImage\" /></html>";
var regex = new Regex("(<img(.+?)id=\"someImage\"(.+?))src=\"([^\"]+)\"");
string output = regex.Replace(
input,
match => match.Groups[1].Value + "src=\"someothervalue\""
);
In the example above there will be 5 groups:
Groups[0] This is the whole match: <img id=\"someImage\" src=\"C:\\logo.png\"
Groups[1] This is everything before the src attribute: <img id=\"someImage\"
Groups[2] and Groups[3] are the (.+?) parts.
Groups[4] is the original value of the src attribute: C:\logo.png
In the example I replace the whole match for the value of Groups[1] and a new src attribute.
Footnote: While regular expressions can sometimes be adequate for the job to manipulate an html document, it is often not the best way. If you know in advance that you are working with xhtml, then you can use XmlDocument + XPath. If it is html, then you can use HtmlAgilityPack.