Saving cURL -w variables to a bash variable

Question

I have a command like

curl -kso /dev/null -w "dns lookup:%{time_namelookup}, tcp:%{time_connect}, ssl:%{time_appconnect} , total:%{time_starttransfer} )\n" https://www.google.com

I want to capture the variables like time_namelookup,time_connect,time_starttransfer which are provided by cURL (as shown in man page snippet below), in a bash variable so that I can use it further.

I tried using read command but that is not helping. I tried something like this

curl -kso /dev/null -w "dns lookup:%{time_namelookup}, tcp:%{time_connect}, ssl:%{time_appconnect} , total:%{time_starttransfer} $(something=%{time_starttransfer})\n" https://www.google.com

but still, the something variable is empty.

Please suggest how should it proceed.

   -w, --write-out <format>
          Defines what to display on stdout after a completed  and  suc‐
          cessful  operation.  The  format  is a string that may contain
          plain text mixed with any number of variables. The string  can
          be  specified  as "string", to get read from a particular file
          you specify it "@filename" and to tell curl to read the format
          from stdin you write "@-".

          The variables present in the output format will be substituted
          by the value or text that curl thinks fit, as described below.
          All  variables are specified as %{variable_name} and to output
          a normal % you just write them as %%. You can output a newline
          by  using  \n,  a carriage return with \r and a tab space with
          \t.

          NOTE: The %-symbol is a special symbol in  the  win32-environ‐
          ment,  where  all  occurrences of % must be doubled when using
          this option.

          The variables available are:

          content_type   The Content-Type of the requested document,  if
                         there was any.

          filename_effective
                         The  ultimate filename that curl writes out to.
                         This is only meaningful  if  curl  is  told  to
                         write  to  a  file  with  the  --remote-name or
                         --output option. It's most useful  in  combina‐
                         tion   with  the  --remote-header-name  option.
                         (Added in 7.25.1)

          ftp_entry_path The initial path curl ended up in when  logging
                         on to the remote FTP server. (Added in 7.15.4)

          http_code      The  numerical  response code that was found in
                         the last retrieved HTTP(S) or FTP(s)  transfer.
                         In  7.18.2 the alias response_code was added to
                         show the same info.

          http_connect   The numerical code that was found in  the  last
                         response  (from  a  proxy)  to  a  curl CONNECT
                         request. (Added in 7.12.4)

          local_ip       The IP address of the local  end  of  the  most
                         recently  done  connection - can be either IPv4
                         or IPv6 (Added in 7.29.0)

          local_port     The local port number of the most recently done
                         connection (Added in 7.29.0)

          num_connects   Number  of  new  connects  made  in  the recent
                         transfer. (Added in 7.12.3)

          num_redirects  Number of redirects that were followed  in  the
                         request. (Added in 7.12.3)

          redirect_url   When  an  HTTP  request  was made without -L to
                         follow redirects, this variable will  show  the
                         actual URL a redirect would take you to. (Added
                         in 7.18.2)

          remote_ip      The remote IP address of the most recently done
                         connection  - can be either IPv4 or IPv6 (Added
                         in 7.29.0)

          remote_port    The remote port number  of  the  most  recently
                         done connection (Added in 7.29.0)

          size_download  The total amount of bytes that were downloaded.

          size_header    The  total  amount  of  bytes of the downloaded
                         headers.

          size_request   The total amount of bytes that were sent in the
                         HTTP request.

          size_upload    The total amount of bytes that were uploaded.
 [...]

Edit: I can see various methods to capture variables but while capturing the output of the given command should not be changed. In short, I'm trying to capture variables without changing the output of the given command.

The trivial way would be to write the output to a file and then parse this file with a suitable program, but if you would show a sample output, we can maybe discuss alternative solutions and perhaps make your attempt with read working. — user1934428
– user1934428, Commented Jan 16, 2020 at 13:45
yeah, writing to a file and parsing it will work but that doesn't seem to be an efficient approach. That approach will not be scalable and will not be a standard solution. — Abhishek saharn
– Abhishek saharn, Commented Jan 16, 2020 at 14:02

Ivan · Accepted Answer · 2020-01-16 14:13:32Z

1

It's not possible just like that coz every command run in its own subshell, create own vars and so on. Try this code:

export somevar=test; curl -kso /dev/null -w "$(export somevar=%{time_connect}; env | grep somevar)\n" https://www.google.com; env | grep somevar

The output(in my case) will be:

somevar=0,004898
somevar=test

So the way to do it is to output directly to var.

somevar=$(curl -kso /dev/null -w "%{time_connect}" https://www.google.com)

$ echo $somevar
0,005093

Or to an array, if you need multiple items.

somearray=( $(curl -kso /dev/null -w "%{time_connect} %{time_namelookup} %{time_appconnect}" https://www.google.com) )

$ echo ${somearray[@]}
0,005095 0,004317 0,165347

edited Jan 16, 2020 at 14:13

answered Jan 16, 2020 at 14:06

Ivan

7,5211 gold badge22 silver badges27 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Abhishek saharn Over a year ago

saving into an array and extracting from there is the only best approach I found. I took all the output into an array variable and echoed it for output, and accessed variables via indices.

Oleg T · Accepted Answer · 2020-01-16 14:12:47Z

0

Here's a way to do it based on Linux bash: Multiple variable assignment

read -r time_namelookup time_connect time_appconnect time_starttransfer<<<$(curl -kso /dev/null -w "%{time_namelookup} %{time_connect} %{time_appconnect} %{time_starttransfer}" https://www.google.com)

The variables:

echo dns lookup:$time_namelookup, tcp:$time_connect, ssl:$time_appconnect, total:$time_starttransfer

Output:

dns lookup:0.004143, tcp:0.044270, ssl:0.112656, total:0.195420

answered Jan 16, 2020 at 14:12

Oleg T

716 bronze badges

Comments

user2587105 · Accepted Answer · 2020-01-16 14:19:48Z

You can use format file and use it in curl.

curl-format.txt:

        time_namelookup:  %{time_namelookup}\n
           time_connect:  %{time_connect}\n
        time_appconnect:  %{time_appconnect}\n
       time_pretransfer:  %{time_pretransfer}\n
          time_redirect:  %{time_redirect}\n
     time_starttransfer:  %{time_starttransfer}\n
                        ----------\n
             time_total:  %{time_total}\n
              http_code: %{http_code}\n

check_stats.sh:

    CURL='/usr/bin/curl'
    CURL_OPTS="-w "@curl-format.txt" -sS"
    URL="https://www.google.com"
    STATS=$(${CURL}  ${CURL_OPTS} ${URL})
    STATS_ARRAY=($( echo $STATS | ${SED} ':a;N;$!ba;s/\r\n/ /g' ))
    time_namelookup=${STATS_ARRAY[1]}
    time_connect=${STATS_ARRAY[3]}
    time_appconnect=${STATS_ARRAY[5]}
    time_pretransfer=${STATS_ARRAY[7]}
    time_redirect=${STATS_ARRAY[9]}
    time_starttransfer=${STATS_ARRAY[11]}
    time_total=${STATS_ARRAY[14]}
    http_code=${STATS_ARRAY[16]}

Collectives™ on Stack Overflow

Saving cURL -w variables to a bash variable

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related