why is awk not reading a shell array?

Question

I am using -v option in awk to read in a shell array. However, it does not seem to be working:

#!/bin/bash

array1=(1 2)

for i in "${array1[@]}"; do
  awk -v vars=${array1[*]} '{for(i=1; i=length(vars); ++i) print vars}'
done

$ ./program
awk: cmd. line:1: fatal: cannot open file `{for(i=1; i=length(vars); ++i) print vars}' for reading (No such file or directory)
awk: cmd. line:1: fatal: cannot open file `{for(i=1; i=length(vars); ++i) print vars}' for reading (No such file or directory)

why does awk appear to attempt to read a file, not elements of the shell array?

${array[*]} expands to two words: 1 and 2. As written, your command is equivalent to awk -v vars=1 2 '{for(i=1; i=length(vars); ++i) print vars}'. So awk understands 2 as the script and for(i=1; i=length(vars); ++i) print vars} as the file to read. Hence the error. — gniourf_gniourf
– gniourf_gniourf, Commented Mar 9, 2015 at 17:09
...and it would not be happening if you weren't violating the cardinal rule of shell scripting: ALWAYS quote your variables unless you have a very specific purpose in mind and fully understand all the caveats. awk -v vars="${array1[*]}" .... — Ed Morton
– Ed Morton, Commented Mar 9, 2015 at 17:11

Jotne · Accepted Answer · 2015-03-09 17:13:32Z

3

Change to:

for i in "${array1[@]}"; do
  awk -v vars="$i" '{for(i=1; i=length(vars); ++i) print vars}' file
done

i do contain the data from the array to use for awk
awk also need a file to read.

edited Mar 9, 2015 at 17:13

answered Mar 9, 2015 at 17:08

Jotne

41.7k13 gold badges54 silver badges58 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

glenn jackman · Accepted Answer · 2015-03-09 17:36:40Z

2

If you want to pass a bash variable into awk and treat it as an awk array, you have to stringify the bash variable and split it in awk:

array1=(foo bar baz "hello world")
(
    IFS=: 
    awk -v str="${array1[*]}" -v sep="[$IFS]" '
        BEGIN {
            n = split(str, a, sep)
            for (i=1; i<=n; ++i) print i, a[i]
        }
    '
)

(using parentheses to localize changes to IFS in a subshell) This outputs

1 foo
2 bar
3 baz
4 hello world

answered Mar 9, 2015 at 17:36

glenn jackman

249k42 gold badges233 silver badges362 bronze badges

Comments

RARE Kpop Manifesto · Accepted Answer · 2022-10-16 13:01:35Z

0

here's an approach for those concerned about typical delimiters showing up in the input data accidentally :

 (export __=$'\37\23\373\3'
  
  mawk '__{ __*= OFS = substr(_, split(__,___,____), FS = "^$" )
          } NF = int(sqrt(_=+($__ = ($__)" :: ")) + !__) substr("",
            OFS = ___[_])' \
              ____="${__}"  \
   __="$( printf "%s${__}" "${signals[@]}" )" <( seq "${#signals[@]}" ))

1 :: EXIT
2 :: HUP
3 :: INT
4 :: QUITQUIT
5 :: ILLILL
6 :: TRAPTRAP
7 :: ABRTABRT
8 :: EMTEMT
9 :: FPEFPEFPE
10 :: KILLKILLKILL
11 :: BUSBUSBUS
12 :: SEGVSEGVSEGV
13 :: SYSSYSSYS
14 :: PIPEPIPEPIPE
15 :: ALRMALRMALRM
16 :: TERMTERMTERMTERM
17 :: URGURGURGURG
18 :: STOPSTOPSTOPSTOP
19 :: TSTPTSTPTSTPTSTP
20 :: CONTCONTCONTCONT
21 :: CHLDCHLDCHLDCHLD
22 :: TTINTTINTTINTTIN
23 :: TTOUTTOUTTOUTTOU
24 :: IOIOIOIO
25 :: XCPUXCPUXCPUXCPUXCPU
26 :: XFSZXFSZXFSZXFSZXFSZ
27 :: VTALRMVTALRMVTALRMVTALRMVTALRM
28 :: PROFPROFPROFPROFPROF
29 :: WINCHWINCHWINCHWINCHWINCH
30 :: INFOINFOINFOINFOINFO
31 :: USR1USR1USR1USR1USR1
32 :: USR2USR2USR2USR2USR2
33 :: ZERRZERRZERRZERRZERR
34 :: DEBUGDEBUGDEBUGDEBUGDEBUG

The idea is to use a 4-byte sequence that's extremely unlikely to occur not just in text, but in ANY type of data ...

  ASCII control byte : d  31 : x1F \037 UNIT-SEP,
          …          : d  19 : x13 \023 DEVICE-CONTROL-3 (XOFF)
  UTF-8 invalid byte : d 251 : xFB \373,
  ASCII control byte : d   3 : x03 \003 END-OF-TEXT

...while avoiding the null byte ("x00 \0"), any of the ones that are either POSIX-spec'ed (or commonly associated with) single-letter escapes ...

x07 - \007 - \a \b \t \n \v \f \r - \015 - x0D

… the ASCII escape byte (x1B "\033"), and the last byte ("xFF \377"),

having none of the bytes are within ± 3 bytes of any other, including wrap-around, plus the added bonus that for every chosen byte, both

31 / 19 / 251 / 3 (the decimal value of the byte ordinal number, and...)

37 / 23 / 373 / 3 (the same value's octal form being interpreted as if it's decimal)

are prime

answered Oct 16, 2022 at 13:01

RARE Kpop Manifesto

3,0096 silver badges15 bronze badges

2 Comments

stack0114106 Over a year ago

where is the 4-byte sequence recommended ?.. any article reference

RARE Kpop Manifesto Over a year ago

@stack0114106 : nowhere - that's just one example i randomly came up with - but feel free to pick anything else that you believe has an extremely low probability of appearing in any type of data, as long as it works for your workflows. i didn't say it was the absolute best approach to things, but at least you can see the reasoning behind some of my choices ( other than the prime part - that was just for fun). the basic approach is simply - pick bytes that people don't remember what it's good for.

Collectives™ on Stack Overflow

why is awk not reading a shell array?

3 Answers 3

Comments

Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related