0

I want to get positional parameters as arguments for my .sh file and I also want to get fields from a text file for awk. I've figured out that I need to use $1-$9 for both and its okay to use the same number $() in awk as a positioning parameter, it still works.

e.g. I call my shell script like this

./myProgram myFile.txt 1 2 3 4

Then within my shell script I want to use awk to refer to fields in a text file like this, specifically 1,2:3,4 the last four fields.

0000000022:trevor:736:1,2:3,4
0000000223:john:73:5,6:7,8
0000002224:eliza:54:9,8:7,6
0000022225:paul:22:5,4:3,2
0000222226:chris:0:1,2:3,4

So I can go through the fields, however when I do because there are two types of field separators it doesn't seem to work.

My shell script so far:

#! /usr/bin/env bash

file="$1"

awk -F'[:,]' -v u1=$5 -v v1=$6 -v u2=$7 -v v2=$8 \ '{ print "u1 =", $u1 }' $1
awk -F'[:,]' -v u1=$5 -v v1=$6 -v u2=$7 -v v2=$8 \ '{ print "v1 =", $v1 }' $1
awk -F'[:,]' -v u1=$5 -v v1=$6 -v u2=$7 -v v2=$8 \ '{ print "u2 =", $u2 }' $1
awk -F'[:,]' -v u1=$5 -v v1=$6 -v u2=$7 -v v2=$8 \ '{ print "v2 =", $v2 }' $1

echo "Argument #1 =" $2
echo "Argument #2 =" $3
echo "Argument #3 =" $4
echo "Argument #4 =" $5

This is the output I get from terminal:

u1 = 1
u1 = 5
u1 = 9
u1 = 5
u1 = 1
v1 = awk: illegal field $(), name "v1"
 input record number 1, file database.txt
 source line number 1
u2 = awk: illegal field $(), name "u2"
 input record number 1, file database.txt
 source line number 1
v2 = awk: illegal field $(), name "v2"
 input record number 1, file database.txt
 source line number 1
Argument #1 = 1
Argument #2 = 2
Argument #3 = 3
Argument #4 = 4

I'm so close yet so far, I'm not sure why I'm not able to go further across the fields using my awk script?

Improve this question
3
  • 2
    By coordinates, I assume you mean fields. There are several problems with your awk commands, but you should first explain better what you're trying to do (using words), and show expected sample output. Commented Mar 30, 2015 at 1:42
  • 1
    possible duplicate of How to use shell variables in awk script Commented Mar 30, 2015 at 1:58
  • possible duplicate of AWK Multiple Field Separators and Variables Commented Mar 30, 2015 at 19:26

1 Answer 1

Reset to default
3

Update: It seems that the OP's problem stemmed from confusion over the distinction between shell parameters ($1, $2, ...) with input-field variables in Awk - which can look the same, but are entirely unrelated.
Specifically, the incorrect assumption was made that if n parameters were passed to the shell script, Awk's input-field numbering would start with n+1.

The following snippet - originally written before the OP added more code to the question - demonstrates the interplay of shell parameters and Awk variables, followed by a detailed explanation.

Specifically, it defines Awk variables fi1 and fi2 based on the values of shell parameters $2 and $3 respectively, which contain 1-based field indices relative to the lines in file $file.

Then, inside the Awk program, the field indices stored in fi1 and fi2 are used first as is (no $ prefix) to print their own value, and then to reference the corresponding input-line fields by prepending $ (variable references in Awk are NOT $-prefixed - the $ is only used to refer to fields).

#!/usr/bin/env bash

file="$1"

awk -F'[:,]' -v fi1=$2 -v fi2=$3 \
  '{ print "Field #" fi1 " + field #" fi2 " =", $fi1 + $fi2 }' "$file"
  • The shell and awk are separate worlds, and do not see each other's variables.
    • You can see environment variables in awk, by accessing the ENVIRON associative array, but you cannot see shell variables.
    • You can implicitly "bake" shell-variable values into an awk program by passing a double-quoted string with shell-variable references - which are expanded before awk sees the program - but that gets confusing quickly and should be avoided.
    • Conversely, the only way to pass values back from awk to the shell is to have awk print to stdout and use a shell command substitution to capture the result in a shell variable.
  • You can pass the values of shell parameters and variables to awk variables using instances of the -v option, as demonstrated above.
    • Here we know that the values are numbers, so we don't strictly need to double-quote the var. references, but it in general it's advisable.
  • Inside awk, its variables are referenced without the $ prefix, and $ is only used to refer to input fields:
    • $fi1, perhaps somewhat confusingly, therefore means: get the input field (prefix $) whose index is stored in awk variable fi1 (by contrast, using fi1 as is, without a prefix, would return the index).
  • Field indices in awk are always 1-based, relative to each line of input.
    • For instance, inside awk, $1 refers to the first field on the current input line, and even though it looks the same as the 1st script / function parameter in the shell, they have absolutely nothing to do with each other.
    • Additionally, $0 contains the entire current input line, and NF contains the number (count) of input fields.
Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

I seem to be getting there not sure what is wrong with my awk line though now @mklement0
@JoshuaVernon: The problem now is that you're seemingly creating Awk variables without a value, which cause field references based on that value - $<varname> - to FAIL (if varname evaluates to the empty string, $ by itself is obviously not a valid field index). Your particular error message suggests that shell parameters $6, $7, and $8 are NOT SET. Note how in your debugging code you're only printing through $5, whereas your Awk commands define variables based on up to $8.
You're right. I've learnt to use $(NF-1) now which seems to work well within an awk

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.