BASH output column formatting

Question

First time posting. HELLO WORLD. Working on my first script that just simply checks if a list of my websites are online and then returns the HTTP code and the amount of time it took to return that to another file on my desktop.

-- THIS SCRIPT WILL BE RUNNING ON MAC OSX --

I would like to amend my script so that it formats its output into 3 neat columns.

currently

#!/bin/bash
file="/Users/USER12/Desktop/url-list.txt"
printf "" > /Users/USER12/Desktop/url-results.txt
while read line
do
    printf "$line" >> /Users/USER12/Desktop/url-results.txt
    printf "\t\t\t\t" >> /Users/USER12/Desktop/url-results.txt
    curl -o /dev/null --silent --head --write-out '%{http_code} %{time_total}' "$line" >> /Users/USER12/Desktop/url-results.txt
    printf "\n" >> /Users/USER12/Desktop/url-results.txt
done <"$file"

which outputs in the following format

google.com              200 0.389
facebook.com                200 0.511
abnormallyLongDomain.com                200 0.786

but i would like to format into neat aligned columns for easy reading

DOMAIN_NAME                 HTTP_CODE   RESPONSE_TIME
google.com                  200         0.389
facebook.com                200         0.511
abnormallyLongDomain.com    200         0.486

Thanks for the help everyone!!

Check out the column command.

Mike Holt
– Mike Holt

2014-09-30 22:08:53 +00:00
Commented Sep 30, 2014 at 22:08 — Mike Holt
– Mike Holt, Commented Sep 30, 2014 at 22:08

John1024 · Accepted Answer · 2014-10-01 18:27:25Z

14

column is very nice. You are, however, already using printf which gives you fine control over the output format. Using printf's features also allows the code to be somewhat simplified:

#!/bin/bash
file="/Users/USER12/Desktop/url-list.txt"
log="/Users/USER12/Desktop/url-results.txt"
fmt="%-25s%-12s%-12s\n"
printf "$fmt" DOMAIN_NAME HTTP_CODE RESPONSE_TIME > "$log"
while read line
do
    read code time < <(curl -o /dev/null --silent --head --write-out '%{http_code} %{time_total}' "$line")
    printf "$fmt" "$line" "$code" "$time" >> "$log"
done <"$file"

With the above defined format, the output looks like:

DOMAIN_NAME              HTTP_CODE   RESPONSE_TIME
google.com               301         0.305
facebook.com             301         0.415
abnormallyLongDomain.com 000         0.000

You can fine-tune the output format, such as spacing or alignment, by changing the fmt variable in the script.

Further Refinements

The above code opens and closes the log file with each loop. This can be avoided as Charles Duffy suggests, simply by using exec to redirect stdout to the log file before the first printf statement:

#!/bin/bash
file="/Users/USER12/Desktop/url-list.txt"
exec >"/Users/USER12/Desktop/url-results.txt"
fmt="%-25s%-12s%-12s\n"
printf "$fmt" DOMAIN_NAME HTTP_CODE RESPONSE_TIME
while read line
do
    read code time < <(curl -o /dev/null --silent --head --write-out '%{http_code} %{time_total}' "$line")
    printf "$fmt" "$line" "$code" "$time"
done <"$file"

Alternatively, as Chepner suggests, the print statements can be grouped:

#!/bin/bash
file="/Users/USER12/Desktop/url-list.txt"
fmt="%-25s%-12s%-12s\n"
{
printf "$fmt" DOMAIN_NAME HTTP_CODE RESPONSE_TIME
while read line
do
    read code time < <(curl -o /dev/null --silent --head --write-out '%{http_code} %{time_total}' "$line")
    printf "$fmt" "$line" "$code" "$time"
done <"$file"
} >"/Users/USER12/Desktop/url-results.txt"

An advantage of grouping is that, after the group, stdout is automatically restored to its normal value.

edited Oct 1, 2014 at 18:27

answered Sep 30, 2014 at 23:00

John1024

115k15 gold badges151 silver badges183 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Charles Duffy Over a year ago

Better, but why not just exec >"$log" once and not need to do any per-command redirects at all? And, of course, you get the efficiency benefits of not re-opening the same file over and over.

chepner Over a year ago

Another way to avoid re-opening "$log" each time you try to write to it: put all commands that will write to it inside one command group: { printf "$fmt" ...; while read line; do ...; done; } < "$file" > "$log".

aLee788 Over a year ago

@CharlesDuffy Thank you for showing me another way to go about this. I have implemented the second version that you have suggested.

clt60 · Accepted Answer · 2014-09-30 23:26:30Z

Shortened a bit

#!/bin/bash

file="./url.txt"
fmt="%s\t%s\t%s\n"
( printf "$fmt" "DOMAIN_NAME" "HTTP_CODE" "RESPONSE_TIME"
while read -r line
do
    printf "$fmt" "$line" $(curl -o /dev/null --silent --head --write-out '%{http_code} %{time_total}' "$line")
done <"$file" ) | column -t > ./out.txt

Don't need redirect every printf but you can enclose the part of your script into (...) and run it in an subshell a redirect it's output. Print every field separated with one tab and use the column command to format it nicely.

Anyway, usually is better don't put filenames (nor headers) into the script and reduce it to

#!/bin/bash

while read -r line
do
    printf "%s\t%s\t%s\n" "$line" $(curl -o /dev/null --silent --head --write-out '%{http_code} %{time_total}' "$line")
done | column -t

and use it like:

myscript.sh < url-list.txt >result.txt

this allows you use your script in pipes, like:

something_produces_urls | myscript.sh | grep 200 > somewhere.txt

Collectives™ on Stack Overflow

BASH output column formatting

2 Answers 2

Further Refinements

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Further Refinements

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related