2

I want to give a value from a file read to a variable, then use it to modify all elements of a column in another file, specifically to shift the entries by some number.

The data file "file_to_change" has two columns of numerical data, one to be shifted in value. Like so:

-1.643465E+01    1.378835E-03
-1.642464E+01    1.242392E-03
-1.641463E+01    1.147501E-03
-1.640462E+01    1.083067E-03
-1.639461E+01    1.042069E-03
-1.638460E+01    1.020041E-03
-1.637459E+01    1.014211E-03

The file "tmp" which contains the shifting value is simply this:

-11.43465

I current have the following code:

file=tmp
var1=$(awk 'NR==1' $file)

echo $var1

awk '{print ($1 - $var1) " " $2}' file_to_change

echo $var1

My result is:

   -11.43465
   0 1.378835E-03
   0 1.242392E-03
   0 1.147501E-03
   0 1.083067E-03
   0 1.042069E-03
   0 1.020041E-03
   0 1.014211E-03
   -11.43465

I imagine this is pretty straightforward, but I can't see where the error is!

Improve this question
2
  • 2
    You can't use shell variables in awk like that ($var1). You need to give it's value to awk variable like: awk -v var1="$var1" '{ ... var1 ... Commented Aug 30, 2017 at 11:00
  • You could pass a variable to awk from shell like this: awk -v 'var1=value' '{print ($1 - $var1) " " $2}' file_to_change (In awk you refer to the variable as var1 and not $var1) Commented Aug 30, 2017 at 11:05

2 Answers 2

Reset to default
1

As others have mentioned, you're not passing the shell variable into awk correctly. The single quotes around the awk body are preventing shell variable expansion. And if you used double quotes around the awk body, then the shell will expand the awk variables "$1" and "$2".

You could do this quoting mess:

awk '{print ($1 - '"$var1"') " " $2}' file_to_change

Or the much cleaner

awk -v var="$var1" '{print ($1 - var) " " $2}' file_to_change

To explain why you get 0 in the first column: in awk, an unset variable is treated as an empty string or, if you're in a "numeric context", the number zero. You have $var1 where the awk variable "var1" is not set. The $ is a kind of operator, returning the value of the column number. So $1 is the value of the first column, $2 the value of the second column, etc. You can use variables there, so x=14; print $x, $(x+1) prints the 14th and 15th column. Since the var1 variable is unset, the expression $1 - $var1 becomes $1 - $0 which becomes (for the first line):

"-1.643465E+01" - "-1.643465E+01    1.378835E-03"

When awk does arithmetic with strings, the first non-numeric character and following characters are simply dropped. To demonstrate:

$ awk 'BEGIN {print "10foo" - "7bar"}'
3
$ awk 'BEGIN {print "1e1foo" + "-700e-2bar"}'
3

So, "-1.643465E+01" - "-1.643465E+01 1.378835E-03"
becomes -1.643465E+01 - -1.643465E+01
hence 0

This can be easily reproduced:

$ seq 10 | paste - - | awk '{print $1 - $novar, $2}'
0 2
0 4
0 6
0 8
0 10
Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

I'd write your code like: awk -v var="$(< tmp)" '{$1 -= var} 1' file_to_change
1

If I understood it, you want to subtract value in file "tmp" from column 1 in file "file_to_change", right?

One way is this:

awk 'NR==FNR{val=$0; nextfile} {$1-=val; print}' tmp file_to_change

First, we read the value from the first line of file "tmp" and skip to the next file.

For each line in "file_to_change" subtract the value from column 1 and print the resulting line.

The result is this:

-5 1.378835E-03
-4.98999 1.242392E-03
-4.97998 1.147501E-03
-4.96997 1.083067E-03
-4.95996 1.042069E-03
-4.94995 1.020041E-03
-4.93994 1.014211E-03
Improve this answer

1 Comment

Note that nextfile is specific to GNU awk

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.