How to display a diff of two files in a Bash script?

Question

I'm trying to compare the contents of two files in a bash script.

local_file=$(cat my_local_file.txt)
remote_file=$(curl -s "http://example.com/remote-file.txt")

if [ local_file == remote_file ]; then
  echo "Files are the same"
else
  echo "Files are different. Here is the diff:"
  diff <(echo "$local_file") <(echo "$remote_file")
fi

When I run the script, I see that I have a syntax error:

./bin/check_files.sh: line 8: syntax error near unexpected token `('
./bin/check_files.sh: line 8: `  diff <(echo "$local_file") <(echo "$remote_file")'

What am I doing wrong? How can I display a diff of these two strings from a bash script?

It's probably a better idea to download the file locally, rather than hold it in memory in Bash. In particular $(...) doesn't preserve trailing newlines or NUL characters, so the contents of local_file and remote_file could potentially not reflect what's on-disk. — dimo414
– dimo414, Commented Jul 10, 2018 at 20:35
@dimo414 Does diff deal with NUL characters? I suspect not. — Barmar
– Barmar, Commented Jul 10, 2018 at 21:49
@Barmar I can't say universally, since there are different implementations, but I'd consider any diff program that fails to do so broken. GNU diff does indeed work with NUL characters, try diff <(printf 'foo\0bar') <(printf 'foo\0\0bar'). — dimo414
– dimo414, Commented Jul 11, 2018 at 0:20
@dimo414 I don't think POSIX requires utilities that process text files to deal with embedded NUL characters. Many GNU utilities try to go beyond the requirements, but it's not something I would generally expect. — Barmar
– Barmar, Commented Jul 11, 2018 at 4:28
@Barmar I'm not sure if this is a canonical source, but it looks like POSIX diff (see "Diff Binary Output Format") explicitly supports binary files. — dimo414
– dimo414, Commented Jul 11, 2018 at 5:32

hek2mgl · Accepted Answer · 2018-07-10 20:36:14Z

5

Process substitution is a bash feature, which is usually not available in /bin/sh which is meant to be POSIX compatible.

Make sure to use the following shebang line if you want to run the script as an executable:

#!/bin/bash

instead of

#!/bin/sh

or use

bash script.sh

instead of

sh script.sh

if you run it like that

To make the script work with POSIX conform shells I would just download the file and compare it against the local file. Remove the downloaded file after the diff.

edited Jul 10, 2018 at 20:36

answered Jul 10, 2018 at 20:27

hek2mgl

159k31 gold badges263 silver badges279 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Gordon Davisson Over a year ago

Process substitution isn't even available in bash if it's run under the name "sh"; therefore, you must explicitly use "bash" instead of "sh" (preferably via the shebang, but running the script with bash scriptname would also work).

codeforester · Accepted Answer · 2018-07-10 21:28:16Z

2

In addition to the <(command) (process substitution) syntax issue, your code if [ local_file == remote_file ] compares the literal strings local_file and remote_file, rather than the content of the variables. You need $local_file and $remote_file to compare the contents. Need to enclose them in double quotes to prevent word splitting issues.

You could do this:

#!/bin/bash

local_file=$(< my_local_file.txt) # this is more efficient than $(cat file)
remote_file=$(curl -s "http://example.com/remote-file.txt")

if [ "$local_file" = "$remote_file" ]; then
  echo "Files are the same"
else
  echo "Files are different. Here is the diff:"
  diff <(printf '%s' "$local_file") <(printf '%s' "$remote_file")
fi

As stated by @dimo414, the limitation here is that the command substitution $(...) removes trailing newlines and that would cause a problem. So, it is better to download the remote file and compare it with the local file:

local_file=my_local_file.txt
curl -s "http://example.com/remote-file.txt" -o remote_file

if diff=$(diff -- "$local_file" remote_file); then
  echo "Files are the same"
else
  echo "Files are different. Here is the diff:"
  printf '%s' "$diff"
fi

edited Jul 10, 2018 at 21:28

answered Jul 10, 2018 at 20:36

codeforester

43.8k21 gold badges122 silver badges159 bronze badges

3 Comments

Ryan Prescott Over a year ago

diff's return value tells you if the inputs match, so you could get by without the cmp call. Something like this should work and be slightly quicker.

local_file=my_local_file.txt curl -s "http://example.com/remote-file.txt" -o remote_file  OUTPUT=$( diff -- "$local_file" "$remote_file" )  if [ $? ];   echo "Files are the same" else   echo "Files are different. Here is the diff:"   echo "$OUTPUT" fi

Ryan Prescott Over a year ago

In your initial description and code, if [ "$a" = "$b" ]; then compares the strings also? curl would download to the file called remote_file, so the cmp line should be if cmp -s -- "$local_file" "remote_file"; then

codeforester Over a year ago

Thanks @RyanPrescott. Updated the answer.

latika bhurani · Accepted Answer · 2018-07-10 20:31:40Z

-1

You can also use the following command:

cmp -b "File_1.txt" "File_2.txt"

answered Jul 10, 2018 at 20:31

latika bhurani

807 bronze badges

1 Comment

Barmar Over a year ago

How does this help when the file data is in variables?

Collectives™ on Stack Overflow

How to display a diff of two files in a Bash script?

3 Answers 3

1 Comment

3 Comments

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

3 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related