1

I am facing an issue with sed in a while-loop.using sed. I want to read the 2nd column of file1, compare it with the content of file2, and if the string is matched, i want to replace the matched string of file1 with file2 string.

I tried with the following code, but it is not returning any output.

cat file1 | while read a b; do
  sed -i "s/$b/$(grep $b file2)/g" file1 > file3;
done 

Example input:

file_1 content:

1 1234
2 8765

file2 content:

12345
34567
87654

Expected output:

1 12345
2 87654
1
  • 1
    You are preforming inplace edits with sed. This does not return any output. So file1 is updated on the fly. Commented Jan 20, 2020 at 10:09

2 Answers 2

1

Your script is very inefficient. Using the while-loop you read each line of file1. This is N operations. Per line you process with the while loop, you reproscess the full file1, making it an N*N process. However, in the sed, you grep file2 constantly. If file2 has M lines, this becomes an N*N*M process. This is very inefficient.

On top of that there are some issues:

  • You updated file1 inplace because you use the -i flag. An inplace update does not provide any output, so file3 will be empty.
  • You are reading file1 with the while-loop and at the same time you update file1 with sed. I don't know how this will react, but I don't believe it is healthy.
  • If $b is not in file2 you would, according to your logic, have a line with only a single column. This is not what you expect.

A fix of your script, would be this:

while read -r a b; do
  c=$(grep "$b" file2)
  [[ "$c" == "" ]] || echo "$a $c"
done < file1 > file3

which is still not efficient, but it is already M*N. The best way is using awk


note: as a novice, always parse your script with http://www.shellcheck.net
note: as a professional, always parse your script with http://www.shellcheck.net

Sign up to request clarification or add additional context in comments.

Comments

1

Could you please try following.

awk 'FNR==NR{a[$2]=$1;next} {for(i in a){if(match($0,"^"i)){print a[i],$0;continue}}}'  file1  file2

Adding a non-one liner form of solution:

awk '
FNR==NR{
  a[$2]=$1
  next
}
{
  for(i in a){
    if(match($0,"^"i)){
      print a[i],$0
      continue
    }
  }
}
'  Input_file1  Input_file2

Explanation: Adding detailed explanation for above code.

awk '                             ##Starting awk code from here.
FNR==NR{                          ##Checking condition if FNR==NR then do following.
  a[$2]=$1                        ##Creating array a whose index is $2 and value is $1.
  next                            ##next will skip all further statements from here.
}
{                                 ##Statements from here will run for 2nd Input_file only.
  for(i in a){                    ##Traversing through array a all elements here.
    if(match($0,"^"i)){           ##Checking condition if current line matches index of current item from array a then do following.
      print a[i],$0               ##Printing array a whose index is i and current line here.
      continue                    ##Again take cursor to for loop.
    }
  }
}
'  Input_file1  Input_file2       ##Mentioning all Input_file names here.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.