AWK - passing generated string as code for awk from command line

Question

I am trying to dynamically print selected fields of a file using awk.

Basically, the question is why this one works:

awk -F';' '{print $3 ";" $4 ";" }' file

but this does not:

awk -F';' '{print '`echo '$3 ";" $4 ";"'`' }' file

My final goal is to encapsulate the above in one command so that I would be able to do

my_cmd ';' 3 6 7 8 file(s)

which should display the fields #3,6,7,8 delimited by ; from file(s).

Editing my post: Found out myself that my problem was echo was inserting of course a new line caracter which was causing issue for awk :o) \c did the trick, also some \" escaping had to do (see below).

awk -F';' '{print '"`echo '$3 \";\" $4 \";\"\c'`"' }' file(s)

Now I'm only left to change it with command which will generate dinamically a string like $2 ";' $5 ";' $6 ";' $9 ";" (number and fields should be the intput) etc. which should go between '{print ' and ' }'

Thanks to cbuckley I found out my one line command: (Issue solved).

cut -d"$1" -f `shift; echo $* | sed 's/[^ 0-9]\{1,\}.*$//;s/[ ]$//;s/[ ]\{1,\}/,/g'` `shift; printf "%s\n" $(echo $*) | grep -v '^[0-9]$'`

here $* are the input parameters and if above would be as an alias or a function in your .rc file named say filterc then the synopsys would be:

filterc delimiter column1 [column2 [column3...]] file1 [file2 [file3...]]

where:

delimiter - one char only

column1..n - a number representing the column

file1..n - files to be filtered, assumption here that files will not have names from numbers only and that all are of same format.

echo and reverse tilde has no meaning inside awk. You should better explain what you're trying to do? — anubhava
– anubhava, Commented Jun 9, 2014 at 10:13
To pass shell variables to awk you need to use the -v parameter. See stackoverflow.com/questions/6373379/using-awk-with-variables — dogbane
– dogbane, Commented Jun 9, 2014 at 10:17
test what you did: echo '{print ...}' (bloody syntax highlight doesn't let me write the full code) — Karoly Horvath
– Karoly Horvath, Commented Jun 9, 2014 at 10:18
Thank you all for your inputs. Looks like found itself what was the issue: echo was adding a \n after it was executed which caused the line to be broken : } was not present to close the statement for awk. Also some tweakinings had to do to finaly get it work awk -F';' '{print '"echo '$3 \";\" $4 \";\"\c'"' }' — vadimbog
– vadimbog, Commented Jun 9, 2014 at 16:26
Software that produces the output you want given a specific sample input set is the starting point towards finding a solution. It is not necessarily a good solution. In this case at a minimum you're opening yourself up to inconsistent echo implementations, arguments that may expand to awk code that causes bizarre errors, complicated quoting issues, potential globbing and word splitting issues, etc. And it's all 100% unnecessary when you can simply pass awk a string of the fields you want printed and let it do that. — Ed Morton
– Ed Morton, Commented Jun 10, 2014 at 0:11

cmbuckley · Accepted Answer · 2014-06-09 10:30:47Z

5

Your command sounds very similar to cut:

cut -d ';' -f 1,3 <<EOT
one;two;three
foo;bar;quux
EOT

one;three
foo;quux

cut -d '-' -f 2,4 <<EOT
one-two-three-four
five-six-seven-eight
EOT

two-four
six-eight

answered Jun 9, 2014 at 10:30

cmbuckley

43k10 gold badges83 silver badges95 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Adrian Frühwirth Over a year ago

Heh, yeah that's preferable (unless you are dealing with multi-character delimiters which cut doesn't do), +1.

Adrian Frühwirth · Accepted Answer · 2014-06-09 10:19:10Z

3

Is this what you are after?

#!/bin/bash

get_columns()
{
    local fs=$1; shift
    local _awk=
    local column

    for column; do
        _awk="${_awk}\$${column},"
    done
    awk -F"$fs" -v OFS="$fs" "{ print ${_awk%,} }"
}

get_columns ';' 1 3 <<EOT
one;two;three
foo;bar;quux
EOT

get_columns '-' 2 4 <<EOT
one-two-three-four
five-six-seven-eight
EOT

-

$ ./t.sh
one;three
foo;quux
two-four
six-eight

answered Jun 9, 2014 at 10:19

Adrian Frühwirth

46.1k10 gold badges63 silver badges71 bronze badges

Comments

Ed Morton · Accepted Answer · 2014-06-09 20:12:33Z

2

function my_cmd {
    fs="$1"
    shift
    eval file="\$$#"
    awk -F"$fs" -v flds="$*" '
        BEGIN{ n=split(flds,f,/ /) }
        {
           for (i=1; i<n; i++)
               printf "%s%s", (i>1?FS:""), $(f[i])
           print ""
        }
    ' "$file"
}

$ cat file
a;b;c;d;e;f;g;h

$ my_cmd ';' 3 6 7 8 file                                                     
c;f;g;h

$ my_cmd ';' 6 3 8 file  
f;c;h

edited Jun 9, 2014 at 20:12

answered Jun 9, 2014 at 19:03

Ed Morton

209k18 gold badges90 silver badges212 bronze badges

2 Comments

vadimbog Over a year ago

How to adjust above if more than one file to give as input?

Ed Morton Over a year ago

Add code above the awk script to separate file names from non-file-names. Since any number can be a file name, I don't know how you plan to separate them.

Collectives™ on Stack Overflow

AWK - passing generated string as code for awk from command line

3 Answers 3

1 Comment

Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related