Create columns using sed data (bash only)

Question

I am relatively new to bash text editing commands and am trying to improve my skills so please bear with me!

I'm looking to write a simple command sequence for locating all SSL certificates on my platform and my ideal output would be three columns for the path, the filename and the expiry.

OK so far I have something like this:

sudo find / -type f \( \( -name "*.pem" -o -name "*.crt" \) ! -path "/unwanted/search/location/*" \) -exec sh -c 'echo {}; true' \; -exec openssl x509 -enddate -noout -in {} \; | sed s/notAfter=// | sed -r 's:/([a-Z0-9._-]*)$:\r\n\1:' | sed 's/GMT/&\n/'

This produces a usable output (am took me some fiddling to get functional). Here is some example output:

/etc/pki/tls/certs
gogs.crt
Feb  8 23:59:59 2018 GMT

/etc/pki/tls/certs
sample.crt
Jan 18 23:59:59 2018 GMT

Finally its question time - what tools should i leverage to reformat that to the following:

/etc/pki/tls/certs  gogs.crt   Feb  8 23:59:59 2018 GMT
/etc/pki/tls/certs  sample.crt Jan 18 23:59:59 2018 GMT

Straight columns are ideal, but not my initial goal - for now I really just wanted to understand how to improve my stream processing skills, surely there must be a way of stating:

Use output from last command, collapse the line breaks, replace with "tab", add linefeed, move onto next.

Is anyone able to offer some advice? Attempts to date have caused some nightmarish output - i would add an example if that wasn't going to add to the confusion!

Thanks readers!

Akshay Hegde · Accepted Answer · 2017-09-05 12:02:21Z

3

You can modify like below inside -exec sh -c ' .... '

-exec sh -c 'file="${0##*/}"; path="${0%/*}"; expiry=$(openssl x509 -enddate -noout -in "$0" | sed "s/notAfter=//");  echo "$path $file $expiry"' {} \;

file="${0##*/}"; - Get filename
path="${0%/*}"; - Get path
expiry=$(openssl x509 -enddate -noout -in "$0" | sed "s/notAfter=//"); - Get expiry date
echo "$path $file $expiry" - print all variable at the end

It will become :

sudo find / -type f \( \( -name "*.pem" -o -name "*.crt" \) ! -path "/unwanted/search/location/*" \) -exec sh -c 'file="${0##*/}"; path="${0%/*}"; expiry=$(openssl x509 -enddate -noout -in "$0" | sed "s/notAfter=//");  echo "$path $file $expiry"  ' {} \;

answered Sep 5, 2017 at 12:02

Akshay Hegde

17.1k2 gold badges25 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

hek2mgl Over a year ago

Side note: I'm not 100% sure where I have this from, but afair $0 is not guaranteed to work like this. Usually you'll use a trick like this: -exec bash -c '...' -- {} \; and then access the filename as $1. ($0 would be -- in that case)

hek2mgl Over a year ago

From bash manual: A positional parameter is a parameter denoted by one or more digits, other than the single digit 0

Akshay Hegde Over a year ago

@hek2mgl: Thanks so much, but I am bit confused personally, from man sh -c string If the -c option is present, then commands are read from string. If there are arguments after the string, they are assigned to the positional parameters, starting with $0 , man7.org/linux/man-pages/man1/sh.1p.html, and I tested few files on centos 6

puppyFlo Over a year ago

This solution, with "| column -t" appended has given me exactly what i hoped for - thank you so much!

hek2mgl Over a year ago

@AkshayHegde I totally understand your confusion. I'm confused too. Plus all currently known shells seem to support your code. But POSIX at least recommends to keep the first param (which ends in $0) for the command name. pubs.opengroup.org/onlinepubs/009695399/utilities/sh.html

|

Ed Morton · Accepted Answer · 2017-09-05 13:35:01Z

3

The following precisely does what you asked for (Use output from last command, collapse the line breaks, replace with "tab", add linefeed, move onto next.):

awk -v RS= -F'\n' -v OFS='\t' '{$1=$1}1'

but adding yet another command to your pipeline isn't a great approach vs just using one script to do it all. If you ask a new question and post the output of the part of your script before the seds pipeline (i.e. the input to sed s/notAfter=// | sed -r 's:/([a-Z0-9._-]*)$:\r\n\1:' | sed 's/GMT/&\n/) as sample input and the expected output you want after processing that input then I'm sure someone can help you with that.

edited Sep 5, 2017 at 13:35

answered Sep 5, 2017 at 13:22

Ed Morton

209k18 gold badges90 silver badges212 bronze badges

1 Comment

puppyFlo Over a year ago

thank you for your advice - i'll need some time to research how awk interprets your instruction but its certainly doing the trick. Very much appreciate your time and advice!

tso · Accepted Answer · 2017-09-06 06:52:36Z

1

your command|awk 'NF{ORS=(NR%3?FS:RS);print}'|column -t

edited Sep 6, 2017 at 6:52

answered Sep 5, 2017 at 12:02

tso

4,9523 gold badges24 silver badges35 bronze badges

4 Comments

hek2mgl Over a year ago

why do you use awk twice?

puppyFlo Over a year ago

This works really great, thank you. I'll have to find some time to properly parse what you've written before I can claim to understand it but the output is ideal.

Ed Morton Over a year ago

In addition to @hek2mgls point, you asked for a command that produced tab-separated output and the above does not produce tab-separated output so how can it work really great? It will also fail when your file names contain spaces.

puppyFlo Over a year ago

I agree that the solution is not ideal, and thanks for pointing out when it would fail. I'm still struggling to interpret it though - it feels hacky even though i don't know what i'm looking at!

Collectives™ on Stack Overflow

Create columns using sed data (bash only)

3 Answers 3

6 Comments

1 Comment

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

6 Comments

1 Comment

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related