7

I am trying to measure server response time for specific requests using a .net 6 client running a HttpClient.

Putting a Stopwatch before the GetAsync call normally returns between 120-140 milliseconds.

Checking Same Url on a browser (both Chrome and FireFox , network tab on developer tools) returns normally after about 30-40 milliseconds.

Looking at the server logs normally shows 20-25 milliseconds. (shows only pure server response time)

I figured the reason for the big gaps is because of in my HttpClient Mesaurements i count also the DNS lookup , TLS handshake and so on.

Using Curl i can get a break down of the times :

time_namelookup:  0.003154s
time_connect:  0.049069s
time_appconnect:  0.121174s
time_pretransfer:  0.121269s
time_redirect:  0.000000s
time_starttransfer:  0.170894s
time_total:  0.171033s

Any way I can perform this kind of a time break down using c# HttpClient ?

I made a little progress using DelegatingHandler and putting a Stopwatch before SendAsync. (got about 5 milliseconds less than total time - my guess its after dns lookup)

I found very little documentation regarding this , any pointers ?

Thanks in advance.

2
  • 1
    How about provide you code. Commented Dec 22, 2022 at 7:57
  • @MichaelMao No specific code is required to answer my question as it generic one for HttpClient Commented Dec 22, 2022 at 9:04

1 Answer 1

8

You can use HttpClient telemetry events as described here:

class Program
{
    private static readonly HttpClient _client = new HttpClient();

    static async Task Main()
    {
        // Instantiate the listener which subscribes to the events. 
        using var listener = new HttpEventListener();

        // Send an HTTP request.
        using var response = await _client.GetAsync("https://github.com/runtime");
    }
}

internal sealed class HttpEventListener : EventListener
{
    // Constant necessary for attaching ActivityId to the events.
    public const EventKeywords TasksFlowActivityIds = (EventKeywords)0x80;

    protected override void OnEventSourceCreated(EventSource eventSource)
    {
        // List of event source names provided by networking in .NET 5.
        if (eventSource.Name == "System.Net.Http" ||
            eventSource.Name == "System.Net.Sockets" ||
            eventSource.Name == "System.Net.Security" ||
            eventSource.Name == "System.Net.NameResolution")
        {
            EnableEvents(eventSource, EventLevel.LogAlways);
        }
        // Turn on ActivityId.
        else if (eventSource.Name == "System.Threading.Tasks.TplEventSource")
        {
            // Attach ActivityId to the events.
            EnableEvents(eventSource, EventLevel.LogAlways, TasksFlowActivityIds);
        }
    }

    protected override void OnEventWritten(EventWrittenEventArgs eventData)
    {
        var sb = new StringBuilder().Append($"{eventData.TimeStamp:HH:mm:ss.fffffff}  {eventData.ActivityId}.{eventData.RelatedActivityId}  {eventData.EventSource.Name}.{eventData.EventName}(");
        for (int i = 0; i < eventData.Payload?.Count; i++)
        {
            sb.Append(eventData.PayloadNames?[i]).Append(": ").Append(eventData.Payload[i]);
            if (i < eventData.Payload?.Count - 1)
            {
                sb.Append(", ");
            }
        }

        sb.Append(")");
        Console.WriteLine(sb.ToString());
    }
}

This produces the following log (I removed long event ids to make it shorter):

08:25:27.6440324 System.Net.Http.RequestStart(scheme: https, host: github.com, port: 443, pathAndQuery: /runtime, versionMajor: 1, versionMinor: 1, versionPolicy: 0)
08:25:27.6782964 System.Net.NameResolution.ResolutionStart(hostNameOrAddress: github.com)
08:25:27.8075834 System.Net.NameResolution.ResolutionStop()
08:25:27.8082958 System.Net.Sockets.ConnectStart(address: InterNetworkV6:28:{1,187,0,0,0,0,0,0,0,0,0,0,0,0,0,0,255,255,140,82,121,3,0,0,0,0})
08:25:27.8805709 System.Net.Sockets.ConnectStop()
08:25:27.8829670 System.Net.Security.HandshakeStart(isServer: False, targetHost: github.com)
08:25:28.1419994 System.Net.Security.HandshakeStop(protocol: 12288)
08:25:28.1431643 System.Net.Http.ConnectionEstablished(versionMajor: 1, versionMinor: 1)
08:25:28.1443727 System.Net.Http.RequestLeftQueue(timeOnQueueMilliseconds: 474,269, versionMajor: 1, versionMinor: 1)
08:25:28.1454417 System.Net.Http.RequestHeadersStart()
08:25:28.1461159 System.Net.Http.RequestHeadersStop()
08:25:28.6777661 System.Net.Http.ResponseHeadersStart()
08:25:28.6783369 System.Net.Http.ResponseHeadersStop()
08:25:28.6826666 System.Net.Http.ResponseContentStart()
08:25:28.8978144 System.Net.Http.ResponseContentStop()
08:25:28.8978472 System.Net.Http.RequestStop()

As you can see, these events contain all information you need: DNS lookup time, connection time, SSL handshake time, and actual request timings.

Update: regarding your concern that with this approach you might receive events not related to the specific web request you are interested in. You can do the correlation using AsyncLocal variable, as documentation mentions here. The idea is simple - you use AsyncLocal variable and set its value to something (such as class holding your timing information) before doing request with HttpClient. Then you perform request. Now, when new event comes in - you check the value of AsyncLocal variable. If it's not null - then this event is related to the current request, otherwise you can ignore it.

Here is extended version of the code above with this approach in mind:

class Program
{
    private static readonly HttpClient _client = new HttpClient();

    static async Task Main()
    {
        using (var listener = new HttpEventListener())
        {
            // we start new listener scope here
            // only this specific request timings will be measured
            // this implementation assumes usage of exactly one HttpEventListener per request
            using var response = await _client.GetAsync("https://github.com/runtime");
            var timings = listener.GetTimings();
            if (timings.Request != null)
                Console.WriteLine($"Total time: {timings.Request.Value.TotalMilliseconds:N0}ms");
            if (timings.Dns != null)
                Console.WriteLine($"DNS: {timings.Dns.Value.TotalMilliseconds:N0}ms");
            if (timings.SocketConnect != null)
                Console.WriteLine($"Socket connect: {timings.SocketConnect.Value.TotalMilliseconds:N0}ms");
            if (timings.SslHandshake != null)
                Console.WriteLine($"SSL Handshake: {timings.SslHandshake.Value.TotalMilliseconds:N0}ms");
            if (timings.RequestHeaders != null)
                Console.WriteLine($"Request headers: {timings.RequestHeaders.Value.TotalMilliseconds:N0}ms");
            if (timings.ResponseHeaders != null)
                Console.WriteLine($"Response headers: {timings.ResponseHeaders.Value.TotalMilliseconds:N0}ms");
            if (timings.ResponseContent != null)
                Console.WriteLine($"Response content: {timings.ResponseContent.Value.TotalMilliseconds:N0}ms");
        }
    }
}

internal sealed class HttpEventListener : EventListener
{
    // Constant necessary for attaching ActivityId to the events.
    public const EventKeywords TasksFlowActivityIds = (EventKeywords)0x80;
    private AsyncLocal<HttpRequestTimingDataRaw> _timings = new AsyncLocal<HttpRequestTimingDataRaw>();

    internal HttpEventListener()
    {
        // set variable here
        _timings.Value = new HttpRequestTimingDataRaw();
    }

    protected override void OnEventSourceCreated(EventSource eventSource)
    {
        // List of event source names provided by networking in .NET 5.
        if (eventSource.Name == "System.Net.Http" ||
            eventSource.Name == "System.Net.Sockets" ||
            eventSource.Name == "System.Net.Security" ||
            eventSource.Name == "System.Net.NameResolution")
        {
            EnableEvents(eventSource, EventLevel.LogAlways);
        }
        // Turn on ActivityId.
        else if (eventSource.Name == "System.Threading.Tasks.TplEventSource")
        {
            // Attach ActivityId to the events.
            EnableEvents(eventSource, EventLevel.LogAlways, TasksFlowActivityIds);
        }
    }

    protected override void OnEventWritten(EventWrittenEventArgs eventData)
    {
        var timings = _timings.Value;
        if (timings == null)
            return; // some event which is not related to this scope, ignore it
        var fullName = eventData.EventSource.Name + "." + eventData.EventName;
        
        switch (fullName)
        {
            case "System.Net.Http.RequestStart":
                timings.RequestStart = eventData.TimeStamp;
                break;
            case "System.Net.Http.RequestStop":
                timings.RequestStop = eventData.TimeStamp;
                break;
            case "System.Net.NameResolution.ResolutionStart":
                timings.DnsStart = eventData.TimeStamp;
                break;
            case "System.Net.NameResolution.ResolutionStop":
                timings.DnsStop = eventData.TimeStamp;
                break;
            case "System.Net.Sockets.ConnectStart":
                timings.SocketConnectStart = eventData.TimeStamp;
                break;
            case "System.Net.Sockets.ConnectStop":
                timings.SocketConnectStop = eventData.TimeStamp;
                break;
            case "System.Net.Security.HandshakeStart":
                timings.SslHandshakeStart = eventData.TimeStamp;
                break;
            case "System.Net.Security.HandshakeStop":
                timings.SslHandshakeStop = eventData.TimeStamp;
                break;
            case "System.Net.Http.RequestHeadersStart":
                timings.RequestHeadersStart = eventData.TimeStamp;
                break;
            case "System.Net.Http.RequestHeadersStop":
                timings.RequestHeadersStop = eventData.TimeStamp;
                break;
            case "System.Net.Http.ResponseHeadersStart":
                timings.ResponseHeadersStart = eventData.TimeStamp;
                break;
            case "System.Net.Http.ResponseHeadersStop":
                timings.ResponseHeadersStop = eventData.TimeStamp;
                break;
            case "System.Net.Http.ResponseContentStart":
                timings.ResponseContentStart = eventData.TimeStamp;
                break;
            case "System.Net.Http.ResponseContentStop":
                timings.ResponseContentStop = eventData.TimeStamp;
                break;
        }
    }

    public HttpRequestTimings GetTimings(){
        var raw = _timings.Value!;
        return new HttpRequestTimings{
            Request = raw.RequestStop - raw.RequestStart,
            Dns = raw.DnsStop - raw.DnsStart,
            SslHandshake = raw.SslHandshakeStop - raw.SslHandshakeStart,
            SocketConnect = raw.SocketConnectStop - raw.SocketConnectStart,
            RequestHeaders = raw.RequestHeadersStop - raw.RequestHeadersStart,
            ResponseHeaders = raw.ResponseHeadersStop - raw.ResponseHeadersStart,
            ResponseContent = raw.ResponseContentStop - raw.ResponseContentStart
        };
    }

    public class HttpRequestTimings{
        public TimeSpan? Request{get;set;}
        public TimeSpan? Dns{get;set;}
        public TimeSpan? SslHandshake{get;set;}        
        public TimeSpan? SocketConnect {get;set;}
        public TimeSpan? RequestHeaders{get;set;}
        public TimeSpan? ResponseHeaders{get;set;}
        public TimeSpan? ResponseContent{get;set;}
    }

    private class HttpRequestTimingDataRaw
    {
        public DateTime? DnsStart { get; set; }
        public DateTime? DnsStop { get; set; }
        public DateTime? RequestStart { get; set; }
        public DateTime? RequestStop { get; set; }
        public DateTime? SocketConnectStart { get; set; }
        public DateTime? SocketConnectStop { get; set; }
        public DateTime? SslHandshakeStart { get; set; }
        public DateTime? SslHandshakeStop { get; set; }
        public DateTime? RequestHeadersStart { get; set; }
        public DateTime? RequestHeadersStop { get; set; }
        public DateTime? ResponseHeadersStart { get; set; }
        public DateTime? ResponseHeadersStop { get; set; }
        public DateTime? ResponseContentStart { get; set; }
        public DateTime? ResponseContentStop { get; set; }
    }
}

In this version I also collect only timings without unrelated info from events. Note that there might be more events for different http requests (for example ones with body).

Sign up to request clarification or add additional context in comments.

7 Comments

Thanks for your answer , I have found that link but it is not specific for HttpClient and maybe even (did not test it) it provides events of the whole system and not only the process.
Updated answer to address this concern.
Thanks for the extra efforts , I will try the code on my scenarios and verify it indeed works. Still i hope/believe there is a simple approach using HttpClient more directly.
Hi , Tested the code with some minor modification for my scenario and it seems to be working quite well. I have added "ConnectionEstablished" event and made new property that is the result of ResponseContentStart - ConnectionEstablished (which is quite similar to the result on server response log value)
Nice to know. I wish there was an easier way but I'm not aware of one.
|

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.